-- Logs begin at Mon 2022-07-11 13:52:40 UTC, end at Mon 2022-07-11 15:00:18 UTC. -- Jul 11 14:30:56.508296 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Jul 11 14:30:56.523390 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Jul 11 14:30:56.526574 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 11 14:30:56 2022] *** Jul 11 14:30:56.526574 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 11 14:30:56.526574 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 11 14:30:56.526574 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: nodename: dv-cinder-master-3-iscsi Jul 11 14:30:56.526574 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: machine: x86_64 Jul 11 14:30:56.526574 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: clock source: unix Jul 11 14:30:56.526574 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: pcre jit disabled Jul 11 14:30:56.526574 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: detected number of CPU cores: 4 Jul 11 14:30:56.526574 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: current working directory: / Jul 11 14:30:56.527660 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: detected binary path: /usr/bin/uwsgi-core Jul 11 14:30:56.527660 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: your processes number limit is 47680 Jul 11 14:30:56.527660 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: your memory page size is 4096 bytes Jul 11 14:30:56.527660 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: detected max file descriptor number: 2048 Jul 11 14:30:56.527660 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: lock engine: pthread robust mutexes Jul 11 14:30:56.527660 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: thunder lock: enabled Jul 11 14:30:56.527660 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Jul 11 14:30:56.527660 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 11 14:30:56.605471 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: Python main interpreter initialized at 0x560da747b9c0 Jul 11 14:30:56.605471 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: python threads support enabled Jul 11 14:30:56.605471 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: your server socket listen backlog is limited to 100 connections Jul 11 14:30:56.605471 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: your mercy for graceful operations on workers is 90 seconds Jul 11 14:30:56.606019 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: mapped 403077 bytes (393 KB) for 2 cores Jul 11 14:30:56.606149 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: *** Operational MODE: preforking *** Jul 11 14:30:56.606715 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: *** uWSGI is running in multiple interpreter mode *** Jul 11 14:30:56.606715 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: spawned uWSGI master process (pid: 105255) Jul 11 14:30:56.607002 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Jul 11 14:30:56.608965 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: spawned uWSGI worker 1 (pid: 105256, cores: 1) Jul 11 14:30:56.608965 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: spawned uWSGI worker 2 (pid: 105257, cores: 1) Jul 11 14:30:56.608965 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 11 14:30:57.333031 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 11 14:30:57.553494 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 11 14:30:58.526847 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 14:30:58.526847 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: from cryptography.utils import int_from_bytes Jul 11 14:30:58.527827 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 14:30:58.527827 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: from cryptography.utils import int_from_bytes Jul 11 14:30:58.713605 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 14:30:58.713605 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: from cryptography.utils import int_from_bytes Jul 11 14:30:58.714507 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 14:30:58.714507 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: from cryptography.utils import int_from_bytes Jul 11 14:31:01.127416 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 11 14:31:01.128508 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 11 14:31:01.128677 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 11 14:31:01.129188 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 11 14:31:01.129826 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 11 14:31:01.129979 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 11 14:31:01.130125 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 11 14:31:01.130272 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 11 14:31:01.130452 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.130697 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 11 14:31:01.130869 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.131055 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 11 14:31:01.131307 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.131566 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.131841 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.132094 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.132304 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.132599 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.132763 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.133113 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.133301 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.133560 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.133716 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.133996 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.134167 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.134360 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.134684 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.134847 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.135179 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.135338 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.135650 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.135835 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.136078 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.136234 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.136496 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.136651 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.136920 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.137096 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.137334 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.137604 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.137850 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.138136 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.138294 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.139170 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.139373 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.139965 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.140168 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.140356 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.140565 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.140764 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.140950 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.141274 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.141487 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.141996 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.142233 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.142419 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.142670 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.142912 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.143188 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.143358 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.143698 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.143866 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.144206 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.144375 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.144701 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.144948 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.145198 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.145458 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.145696 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.145961 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.146196 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.146492 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.146693 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.146985 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.147216 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.147580 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.147751 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.148063 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.148243 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.148605 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.148783 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.149092 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.149264 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.149661 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.149848 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.150005 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.150336 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.150530 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.150849 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.151005 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.151284 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.151440 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.151706 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.151991 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.152257 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.152539 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.152696 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.153017 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.153174 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.153453 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.153631 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.153872 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.154110 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.154793 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.155151 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.155342 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.155625 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.155778 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.156058 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.156214 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.156490 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.156661 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.156849 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.157124 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.157283 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.157583 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.157751 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.158025 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.158236 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.158525 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.158683 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.158948 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.159125 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.159366 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.159547 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.159785 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.160061 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.160229 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.160556 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.160711 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.160973 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.161141 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.161384 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.161562 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.161813 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.161976 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.162239 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.162416 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.162764 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.163045 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.163194 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.163470 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.163638 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.163948 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.164122 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.72 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.164398 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.72 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.164569 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.164815 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.164980 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.165365 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.165533 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.165805 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.165957 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.166224 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.166376 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.166686 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.166860 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.167115 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.167362 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.167516 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.167843 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.168023 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.168309 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.168462 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.168764 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.168930 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.169208 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.169444 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.72 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.169619 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.169890 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.72 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.170058 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.170350 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.170548 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.170786 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.170985 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.171214 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.171496 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.171650 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.171976 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.172144 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.172398 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.172570 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.172913 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.173068 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.173320 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.173535 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.173724 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.174037 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.174193 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.174546 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.174703 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.174980 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.175136 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.175408 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.175625 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.175863 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.176098 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.176255 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.176510 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.176667 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.176981 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.177135 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.177459 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.177614 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.177922 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.178076 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.178347 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.178606 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.178837 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.178991 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.179273 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.179474 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.179722 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.179964 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.180244 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.180458 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.180799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.180956 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.181271 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.181427 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.181694 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.181849 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.182119 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.182327 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.182588 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.182861 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.183090 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.183373 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.183548 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.183855 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.184010 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.184309 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.184464 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.184735 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.184899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.185172 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.185434 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.185656 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.185953 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.186111 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.186462 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.186621 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.186927 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.187106 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.187360 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.187512 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.187788 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.188114 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.188268 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.188592 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.189044 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.189281 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.189482 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.189790 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.189960 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.190275 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.190462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.190766 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.190919 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.191190 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.191396 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.191631 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.191898 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.192083 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.192387 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.192558 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.192882 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.193089 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.193384 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.193542 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.193852 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.194022 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.194316 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.194502 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.194783 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.194957 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.195217 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.195385 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.195649 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.195831 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.196144 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.196297 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.196573 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.196742 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.197009 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.197234 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.197429 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.197724 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.197892 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.198201 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.198356 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.198708 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.198915 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.199195 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.199364 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.199642 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.199797 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.200087 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.200243 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.200523 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.200692 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.201884 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.202216 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.202370 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:31:01.202703 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.202900 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.203194 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.203448 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.203662 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.203983 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.204140 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.204511 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.204702 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.204957 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.205109 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.205409 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.205562 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.205862 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.206031 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.206347 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.206548 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.206815 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.207002 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.207255 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.207537 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.207692 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.208015 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.208171 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.208499 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.208669 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.208981 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.209189 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.209464 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.209634 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.209900 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.210069 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.210335 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.210578 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.210800 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.211092 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.211251 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.211588 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.211765 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.212077 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.212352 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.212576 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.212794 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.213083 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.213240 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.213576 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.213734 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.214061 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.214232 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.214605 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.214778 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.215044 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.215200 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.215481 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.215705 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.215907 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.216206 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.216373 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.216770 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.216928 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.217263 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.217417 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.217712 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.217880 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.218172 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.218341 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.218674 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.218833 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.219135 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.219291 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.219615 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.219794 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.220063 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.220218 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.220505 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.220670 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.220947 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.221167 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.221387 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.221678 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.221847 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.222165 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.222360 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.222959 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.223125 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.223255 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.223479 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.223770 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.224040 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.224335 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.224595 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.224809 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.225173 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.225315 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.225692 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.225832 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.226289 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.226464 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.226730 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.226885 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.227212 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.227405 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.227693 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.227972 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.228166 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.228546 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.228807 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.229108 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.229249 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.229669 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.229816 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.230138 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.230296 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.230675 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.230879 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.231162 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.231319 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.231675 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.231860 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.232686 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.232860 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.233138 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.233295 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.233589 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.233748 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.234028 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.234182 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.234473 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.234674 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.234934 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.235188 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.235359 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.235704 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.235864 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.236160 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.236330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.236597 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.236751 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.237023 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.237219 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.237468 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.237716 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.237998 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.238266 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.238455 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.238784 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.238963 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.239266 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.239422 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.239705 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.239911 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.240164 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.240450 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.240610 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.240808 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.241002 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.241307 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.241464 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.241794 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.241952 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.242294 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.242484 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.242799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.242985 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.243252 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.243421 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.243714 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.243870 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.244155 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.244325 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.244600 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.244814 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.245038 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.245343 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.245502 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.245850 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.246023 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.246290 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.246504 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.246775 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.246956 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.247208 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.247389 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.247678 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.247946 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.248808 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.248981 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.249268 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.249425 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.249704 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.249886 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.250181 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.250403 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.250662 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.250947 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.251104 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.251424 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.251582 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.251907 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.252064 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.252381 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.252553 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.252829 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.253037 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.253262 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.253550 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.253725 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.254033 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.254191 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.254536 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.254695 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.254972 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.255129 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.255403 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.255616 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.255844 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.256126 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.256294 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.256649 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.256808 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.257078 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.257246 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.257551 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.257724 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.258044 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.258211 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.258589 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.258827 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.259056 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.259236 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.259503 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.259737 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.259969 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.260248 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.260495 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.260774 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.260931 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.261257 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.261428 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.261769 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.261940 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.262210 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.262365 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.262708 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.262886 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.263155 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.263311 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.263629 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.263847 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.264064 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.264353 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.264508 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.264849 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.265017 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.265309 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.265478 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.265759 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.265915 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.266190 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.266396 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.266667 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.266972 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.267140 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.267463 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.267627 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.267897 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.268126 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.268376 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.268533 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.268882 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.269037 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.269381 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.269539 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.269827 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.269983 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.270291 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.270545 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.270792 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.270948 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.271230 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.271418 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.271688 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.271916 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.272138 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.272431 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.272610 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.272951 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.273193 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.273430 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.273595 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.273914 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.274073 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.274354 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.274559 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.274817 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.274971 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.275270 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.275476 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.275717 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.276003 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.276158 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.276508 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.276682 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.276946 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.277115 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.277376 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.277592 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.277852 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.278019 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.278346 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.278532 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.278834 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.278990 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.279882 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.280040 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.280337 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.280492 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.280825 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.280994 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.281270 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.281437 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.281742 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.281897 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.282183 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.282338 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.282651 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.282842 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.283093 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.283347 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.283514 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.283844 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.284003 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.284324 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.284494 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.284810 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.285096 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.285367 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.285547 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.285818 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.285976 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.286316 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.286550 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.286852 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.287009 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.287298 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.287454 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.287766 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.287923 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.288228 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.288384 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.288730 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.288897 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.289199 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.289380 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.289666 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.289836 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.290128 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.290284 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.290600 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.290828 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.291087 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.291256 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.291527 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.291841 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.292010 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.292329 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.292499 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.292769 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.292922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.293203 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.293358 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.293644 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.293815 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.294076 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.294308 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.294523 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.295412 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.295584 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.295913 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.296073 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.296353 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.296509 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.296787 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.296943 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.297250 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.297415 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.297693 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.297893 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.298132 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.298400 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.298691 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.298956 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.299113 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.299438 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.299595 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.299927 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.300087 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.300420 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.300576 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.300901 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.301060 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.301391 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.301549 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.301824 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.301978 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.302253 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.302407 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.302757 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.302927 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.303199 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.303385 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.303635 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.304022 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.304248 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.304455 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.304765 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.304924 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.305249 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.305430 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.305704 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.305855 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.306141 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.306295 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.306612 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.306828 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.307044 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.307358 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.307520 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.307848 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.308003 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.308275 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.308441 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.308715 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.308945 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.309208 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.309413 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.309703 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.309927 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.310134 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.310482 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.310706 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.310995 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.311220 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.311506 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.311663 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.311978 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.312134 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.312418 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.312651 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.312877 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.313040 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.313316 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.313470 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.313749 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.313946 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.314187 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.314510 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.314668 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.314962 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.315119 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.315458 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.315633 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.315920 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.316074 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.316366 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.316522 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.316799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.316952 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.317232 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.317429 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.317674 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.317945 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.318100 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.318416 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.318621 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.318928 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.319072 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.319399 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.319619 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.319961 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.320120 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.320440 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.320670 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.320958 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.321113 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.321428 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.321586 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.321919 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.322190 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.322397 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.322581 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.322913 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.323071 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.323407 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.323596 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.323855 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.324009 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.324304 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.324469 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.324811 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.324982 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.325261 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.325417 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.325717 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.325910 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.326217 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.326672 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.326934 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.327155 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.327362 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.327661 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.327833 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.328153 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.328386 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.328645 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.328799 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.329159 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.329331 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.329631 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.329864 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.330138 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.330297 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.330642 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.330878 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.331107 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.331260 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.331606 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.331761 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.332105 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.332278 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.332552 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.332712 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.332993 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.333154 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.333429 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.333583 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.333865 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.334058 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.334306 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.334582 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.334791 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.335092 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.335274 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.335583 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.335739 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.336049 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.336219 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.336501 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.336658 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.336953 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.337122 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.337407 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.337591 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.337837 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.337991 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.338319 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.338522 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.338785 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.339028 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.339197 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.339511 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.339668 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.339983 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.340217 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.340497 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.340672 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.340977 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.341135 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.341441 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.342194 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.342462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.342665 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.342966 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.343221 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.343396 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.343729 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.343887 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.344206 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.344390 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.344759 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.344993 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.345158 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.345303 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.345584 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.345772 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.346091 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.346266 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.346625 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.346783 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.347112 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.347255 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.347576 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.347757 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.348014 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.348169 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.348548 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.348718 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.349019 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.349188 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.349457 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.349612 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.349898 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.350109 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.350420 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.350724 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.350874 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.351075 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.351367 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.351561 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.351810 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.352114 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.352272 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.352583 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.352854 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.353131 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.353304 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.353672 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.353843 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.354108 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.354264 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.354583 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.354803 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.355014 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.355315 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.355473 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.355800 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.355957 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.356277 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.356434 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.356708 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.356879 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.357197 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.357906 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.358202 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.358358 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.358677 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.358861 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.359138 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.359366 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.359711 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.359878 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.360147 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.360352 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.360653 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.360830 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.361215 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.361387 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.361647 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.361803 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.362080 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.362257 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.362592 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.362826 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.363084 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.363331 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.363536 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.363836 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.363991 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.364314 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.364472 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.364791 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.364969 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.365249 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.365418 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.365705 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.365863 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.366154 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.366324 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.366627 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.366782 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.367057 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.367286 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.367502 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.367835 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.368047 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.368350 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.368509 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.368830 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.369002 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.369318 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.369477 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.369798 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.369956 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.370243 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.370412 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.370796 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.370956 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.371279 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.371456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.371712 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.371882 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.372150 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.372304 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.372628 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.372837 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.373159 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.373337 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.373595 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.373749 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.374054 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.374373 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.374622 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.374891 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.375044 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.375336 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.375511 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.375770 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.375937 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.376202 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.376375 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.376635 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.376891 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.377048 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.377374 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.377598 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.377897 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.378052 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.378367 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.378569 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.378853 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.379088 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.379275 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.379584 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.379743 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.380068 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.380225 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.380554 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.380781 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.381063 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.381232 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.381530 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.381713 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.381962 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.382119 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.382461 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.382621 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.382924 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.383135 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.383409 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.383597 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.383863 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.384091 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.384333 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.384609 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.384791 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.385298 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.385508 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.385713 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.385914 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.386202 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.386473 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.386710 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.386972 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.387132 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.387452 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.387624 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.387953 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.388107 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.388474 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.388635 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.388936 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.389106 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.389369 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.389602 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.389759 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.390095 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.390265 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.390619 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.390778 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.391098 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.391309 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.391569 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.391725 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.392022 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.392188 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.392451 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.392645 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.392900 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.393178 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.393396 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.393687 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.393844 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.394176 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.394332 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.394637 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.394809 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.395112 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.395270 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.395538 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.395737 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.395998 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.396256 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.396427 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.396755 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.396923 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.397264 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.397441 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.397701 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.397945 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.398137 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.398489 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.398728 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.399016 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.399174 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.399489 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.399656 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.399923 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.400092 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.400363 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.400532 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.400803 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.401019 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.401303 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.401528 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.401787 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.402007 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.402222 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.402566 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.402727 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.403055 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.403213 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.403538 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.403890 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.404567 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.404869 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.405025 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.405310 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.405495 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.405755 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.406001 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.406180 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.406569 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.406729 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.407072 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.407286 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.407556 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.407742 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.407992 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.408160 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.408434 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.408624 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.408895 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.409151 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.409391 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.409673 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.409886 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.410187 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.410361 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.410707 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.410885 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.411197 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.411367 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.411681 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.411867 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.412171 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.412328 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.412643 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.412799 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.413137 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.413313 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.413613 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.413772 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.414078 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.414234 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.414647 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.414814 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.415133 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.415289 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.415577 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.415732 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.416012 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.416168 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.416446 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.416663 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.416882 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.417175 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.417332 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.417666 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.417824 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.418146 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.418305 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.418654 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.418812 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.419140 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.419299 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.419634 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.420279 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.420620 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.420780 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.421077 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.421235 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.421555 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.421729 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.422003 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.422159 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.422519 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.422696 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.423026 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.423196 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.423524 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.423683 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.423991 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.424150 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.424485 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.424657 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.424954 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.425122 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.425397 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.425555 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.425842 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.426011 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.426287 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.426464 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.426748 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.426982 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.427152 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.427467 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.427682 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.427997 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.428156 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.428436 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.428709 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.428873 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.429076 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.429364 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.429586 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.429815 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.430084 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.430319 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.430640 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.430800 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.431131 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.431463 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.431704 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.431966 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.432122 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.432402 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.432611 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.432852 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.433122 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.433280 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.433607 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.433764 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.434110 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.434286 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.434605 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.434780 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.435040 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.435197 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.435499 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.435694 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.435944 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.436206 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.436365 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.436743 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.437026 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.437199 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.437530 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.437690 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.438005 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.438163 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.438463 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.438647 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.438930 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.439085 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.439402 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.439571 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.439871 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.440048 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.440311 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.440553 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.440711 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.441038 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.441222 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.441523 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.441677 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.441995 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.442150 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.442479 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.442664 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.442897 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.443063 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.443333 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.443582 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.443751 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.444094 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.444250 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.444583 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.444739 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.445027 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.445198 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.445512 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.445669 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.445995 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.446135 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.446510 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.446689 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.446998 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.447169 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.447451 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.447610 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.447924 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.448082 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.448382 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.448541 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.448836 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.448993 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.449353 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.449514 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.449855 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.450012 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.450346 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.450570 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.450868 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.451026 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.451605 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.451764 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.452092 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.452275 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.452610 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.452847 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.453124 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.453281 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.453618 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.453791 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.454135 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.454343 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.454660 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.454817 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.455097 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.455317 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.455537 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.455819 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.455989 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.456309 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.456465 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.456777 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.457119 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.457277 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.457589 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.457747 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.458070 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.458212 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.458614 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.458825 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.459151 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.459309 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.459631 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.459787 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.460113 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.460282 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.460558 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.460724 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.460995 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.461151 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.461434 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.461590 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.461867 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.462089 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.462354 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.462607 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.462848 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.463111 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.463315 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.463630 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.463787 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.464110 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.464268 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.464606 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.464767 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.465058 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.465213 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.465505 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.465663 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.465943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.466338 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.467081 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.467314 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.467509 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.467781 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.468010 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.468246 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.468554 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.468724 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.469052 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.469211 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.469537 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.469697 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.470124 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.470281 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.470611 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.470771 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.471063 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.471242 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.471509 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.471665 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.471958 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.472116 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.472443 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.472636 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.472943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.473099 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.473394 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.473639 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.473910 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.474116 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.474271 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.474639 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.474821 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.475126 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.475304 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.475615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.475774 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.476080 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.476233 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.476507 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.476691 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.476966 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.477107 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.477508 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.477664 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.477957 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.478111 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.478397 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.478650 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.478895 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.479053 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.479324 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.479518 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.479760 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.480023 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.480184 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.480494 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.480664 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.480982 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.481139 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.481444 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.481598 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.481878 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.482058 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.482384 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.482629 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.482867 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.483053 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.483309 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.483554 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.483722 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.484040 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.484198 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.484518 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.484688 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.484998 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.485157 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.485423 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.485588 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.485852 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.486054 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.486285 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.486608 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.486768 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.487095 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.487270 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.487558 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.487713 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.488019 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.488175 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.488456 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.488624 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.488928 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.489140 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.489371 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.489727 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.489886 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.490192 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.490598 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.490759 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.491108 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.491278 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.491550 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.491717 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.491982 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.492173 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.492448 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.492717 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.492898 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.493209 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.493377 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.493695 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.493853 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.494158 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.494335 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.494621 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.494795 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.495094 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.495251 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.495549 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.495720 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.495994 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.496210 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.496421 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.496714 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.496883 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.497206 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.497364 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.497671 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.498356 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.498661 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.498833 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.499145 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.499303 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.499599 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.499756 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.500031 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.500206 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.500473 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.500680 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.500944 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.501158 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.501382 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.501667 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.501823 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.502157 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.502313 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.502667 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.502903 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.503151 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.503310 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.503607 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.503774 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.504041 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.504237 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.504494 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.504745 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.504947 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.505270 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.505430 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.505762 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.505920 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.506242 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.506402 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.506768 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.506946 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.507213 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.507382 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.507657 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.507824 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.508090 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.508315 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.508469 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105257) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 11 14:31:01.508789 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.509330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.509751 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.510204 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.510686 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.511102 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.511528 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.511959 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.512385 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.512817 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.513263 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.513768 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.514213 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.514693 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.515132 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.515761 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.516184 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.516599 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.517012 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.517433 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.517847 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.518266 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.518711 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.519138 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.519570 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.519980 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.520388 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.520804 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.521238 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.521646 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.522065 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.522520 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.522940 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.523375 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.523786 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.524198 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.524609 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.525018 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.525475 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.525883 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.526299 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.526773 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.527214 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.527632 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.528072 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.528490 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.528916 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.529854 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.530306 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.530774 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.531207 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.531624 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.532061 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.532471 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.532885 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.533311 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.533726 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.534163 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.534616 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.535047 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.535472 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.535920 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.536352 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.536773 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.537200 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.537620 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.538036 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.538542 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.538960 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.539375 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.539819 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.540260 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.540690 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.541107 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.541552 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.541988 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.542421 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.542873 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.543310 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.543770 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.544230 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.544686 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.545157 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.545587 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.546035 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.546487 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.546893 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:31:01.547236 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105256) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 11 14:31:01.581092 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO dbcounter [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] Registered counter for database nova_api Jul 11 14:31:01.612765 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO dbcounter [None req-06629404-a8e9-4e5b-99ab-2fa28acfd9ce None None] Registered counter for database nova_api Jul 11 14:31:01.705608 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_db.sqlalchemy.engines [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105257) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 14:31:01.713594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG dbcounter [-] [105257] Writer thread running {{(pid=105257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 14:31:01.735850 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG oslo_db.sqlalchemy.engines [None req-06629404-a8e9-4e5b-99ab-2fa28acfd9ce None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105256) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 14:31:01.743284 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG dbcounter [-] [105256] Writer thread running {{(pid=105256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 14:31:01.773110 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.context [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),14f12807-d728-487f-b4ea-df7168086c51(cell1) {{(pid=105257) load_cells /opt/stack/nova/nova/context.py:464}} Jul 11 14:31:01.777519 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:31:01.779333 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:31:01.780313 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:31:01.782561 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:31:01.784656 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO dbcounter [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] Registered counter for database nova_cell0 Jul 11 14:31:01.788929 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO dbcounter [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] Registered counter for database nova_cell1 Jul 11 14:31:01.797926 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_db.sqlalchemy.engines [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105257) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 14:31:01.801264 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_db.sqlalchemy.engines [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105257) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 14:31:01.805506 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG dbcounter [-] [105257] Writer thread running {{(pid=105257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 14:31:01.807410 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.context [None req-06629404-a8e9-4e5b-99ab-2fa28acfd9ce None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),14f12807-d728-487f-b4ea-df7168086c51(cell1) {{(pid=105256) load_cells /opt/stack/nova/nova/context.py:464}} Jul 11 14:31:01.811895 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG oslo_concurrency.lockutils [None req-06629404-a8e9-4e5b-99ab-2fa28acfd9ce None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105256) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:31:01.813639 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG oslo_concurrency.lockutils [None req-06629404-a8e9-4e5b-99ab-2fa28acfd9ce None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105256) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:31:01.814631 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG oslo_concurrency.lockutils [None req-06629404-a8e9-4e5b-99ab-2fa28acfd9ce None None] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105256) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:31:01.816929 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG oslo_concurrency.lockutils [None req-06629404-a8e9-4e5b-99ab-2fa28acfd9ce None None] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105256) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:31:01.819156 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO dbcounter [None req-06629404-a8e9-4e5b-99ab-2fa28acfd9ce None None] Registered counter for database nova_cell0 Jul 11 14:31:01.823417 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO dbcounter [None req-06629404-a8e9-4e5b-99ab-2fa28acfd9ce None None] Registered counter for database nova_cell1 Jul 11 14:31:01.832816 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG oslo_db.sqlalchemy.engines [None req-06629404-a8e9-4e5b-99ab-2fa28acfd9ce None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105256) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 14:31:01.838665 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG dbcounter [-] [105256] Writer thread running {{(pid=105256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 14:31:01.839346 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG oslo_db.sqlalchemy.engines [None req-06629404-a8e9-4e5b-99ab-2fa28acfd9ce None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105256) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 14:31:01.956927 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG dbcounter [-] [105257] Writer thread running {{(pid=105257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 14:31:01.968202 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO dbcounter [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] Registered counter for database nova_cell0 Jul 11 14:31:01.979266 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_db.sqlalchemy.engines [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105257) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 14:31:01.984172 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG dbcounter [-] [105257] Writer thread running {{(pid=105257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 14:31:01.986034 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG dbcounter [-] [105256] Writer thread running {{(pid=105256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 14:31:01.996184 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO dbcounter [None req-06629404-a8e9-4e5b-99ab-2fa28acfd9ce None None] Registered counter for database nova_cell0 Jul 11 14:31:02.003765 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG oslo_db.sqlalchemy.engines [None req-06629404-a8e9-4e5b-99ab-2fa28acfd9ce None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105256) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 14:31:02.009229 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG dbcounter [-] [105256] Writer thread running {{(pid=105256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 14:31:02.328241 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: WARNING keystonemiddleware.auth_token [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 11 14:31:02.334867 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: WARNING keystonemiddleware.auth_token [None req-06629404-a8e9-4e5b-99ab-2fa28acfd9ce None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 11 14:31:02.335686 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: WARNING keystonemiddleware.auth_token [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 11 14:31:02.342991 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: WARNING keystonemiddleware.auth_token [None req-06629404-a8e9-4e5b-99ab-2fa28acfd9ce None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 11 14:31:02.686725 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: WARNING keystonemiddleware.auth_token [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 11 14:31:02.692616 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: WARNING keystonemiddleware.auth_token [None req-06629404-a8e9-4e5b-99ab-2fa28acfd9ce None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 11 14:31:02.692777 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: WARNING keystonemiddleware.auth_token [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 11 14:31:02.697955 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560da747b9c0 pid: 105257 (default app) Jul 11 14:31:02.698656 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: WARNING keystonemiddleware.auth_token [None req-06629404-a8e9-4e5b-99ab-2fa28acfd9ce None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 11 14:31:02.700098 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:31:02.702064 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] 10.222.0.72 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.002549 Jul 11 14:31:02.702506 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 1/1] 10.222.0.72 () {50 vars in 788 bytes} [Mon Jul 11 14:31:02 2022] GET /compute/v2.1/ => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:31:02.704759 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x560da747b9c0 pid: 105256 (default app) Jul 11 14:32:25.109671 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:32:25.110762 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001476 Jul 11 14:32:25.111950 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 2/2] 10.222.0.72 () {54 vars in 930 bytes} [Mon Jul 11 14:32:25 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:32:25.140264 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG dbcounter [-] [105256] Writing DB stats nova_api:SELECT=3 {{(pid=105256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:32:25.142020 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG dbcounter [-] [105256] Writing DB stats nova_cell0:SELECT=3 {{(pid=105256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:32:25.144348 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG dbcounter [-] [105256] Writing DB stats nova_cell1:SELECT=3 {{(pid=105256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:32:25.146207 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG dbcounter [-] [105256] Writing DB stats nova_cell0:SELECT=3 {{(pid=105256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:32:26.604151 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-b57c0c4b-3e54-4249-9416-c036c9a33d93 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:32:26.643896 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-b57c0c4b-3e54-4249-9416-c036c9a33d93 admin admin] 10.222.0.72 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 1.523394 Jul 11 14:32:26.645306 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 1/3] 10.222.0.72 () {60 vars in 1291 bytes} [Mon Jul 11 14:32:25 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 1527 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:32:29.485650 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:32:29.487946 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-62e2d258-9e95-47ad-8944-5d67e517aa13 None None] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002639 Jul 11 14:32:29.489546 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 3/4] 10.222.0.72 () {54 vars in 930 bytes} [Mon Jul 11 14:32:29 2022] GET /compute/v2.1 => generated 388 bytes in 6 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:32:30.170338 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-4167cff8-7411-4e0e-8258-99b1110b4fc7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "c1", "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 256, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "disk": 1, "vcpus": 1, "swap": 0, "name": "cirros256"}} {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:32:30.198320 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:32:30.198320 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: warnings.warn(msg) Jul 11 14:32:30.237854 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-4167cff8-7411-4e0e-8258-99b1110b4fc7 admin admin] 10.222.0.72 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.739386 Jul 11 14:32:30.238639 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 2/5] 10.222.0.72 () {64 vars in 1282 bytes} [Mon Jul 11 14:32:29 2022] POST /compute/v2.1/flavors => generated 407 bytes in 741 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:32:30.256157 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG dbcounter [-] [105257] Writing DB stats nova_api:SELECT=3 {{(pid=105257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:32:30.257201 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG dbcounter [-] [105257] Writing DB stats nova_cell1:SELECT=3 {{(pid=105257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:32:30.259419 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG dbcounter [-] [105257] Writing DB stats nova_cell0:SELECT=3 {{(pid=105257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:32:30.263086 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG dbcounter [-] [105257] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:32:30.524790 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-289801f5-b98a-457e-a54c-168ca4d68209 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:32:30.528318 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:32:30.528318 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: warnings.warn(msg) Jul 11 14:32:30.607422 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-289801f5-b98a-457e-a54c-168ca4d68209 admin admin] 10.222.0.72 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.359961 Jul 11 14:32:30.623244 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 4/6] 10.222.0.72 () {64 vars in 1335 bytes} [Mon Jul 11 14:32:30 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:32:33.049160 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-4167cff8-7411-4e0e-8258-99b1110b4fc7 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:32:33.050866 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-4167cff8-7411-4e0e-8258-99b1110b4fc7 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002203 Jul 11 14:32:33.051492 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 3/7] 10.222.0.72 () {54 vars in 930 bytes} [Mon Jul 11 14:32:33 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:32:33.891695 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-8c6ed726-889f-4da0-878b-c88e0f2b0ae6 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "name": "ds512M", "rxtx_factor": 1.0, "swap": 0, "vcpus": 1, "id": "d1", "OS-FLV-EXT-DATA:ephemeral": 0, "disk": 5, "os-flavor-access:is_public": true}} {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:32:33.925494 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-manage:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:32:33.925494 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: warnings.warn(msg) Jul 11 14:32:33.956647 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-8c6ed726-889f-4da0-878b-c88e0f2b0ae6 admin admin] 10.222.0.72 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.898292 Jul 11 14:32:33.957909 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 5/8] 10.222.0.72 () {64 vars in 1282 bytes} [Mon Jul 11 14:32:33 2022] POST /compute/v2.1/flavors => generated 404 bytes in 901 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:32:33.971228 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-15760b3e-8bb0-41f1-b1e7-fc8a1bc73b70 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:32:33.974109 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-flavor-extra-specs:create": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:32:33.974109 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: warnings.warn(msg) Jul 11 14:32:34.021342 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-15760b3e-8bb0-41f1-b1e7-fc8a1bc73b70 admin admin] 10.222.0.72 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.055171 Jul 11 14:32:34.022632 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 4/9] 10.222.0.72 () {64 vars in 1335 bytes} [Mon Jul 11 14:32:33 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:32:36.345065 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-8c6ed726-889f-4da0-878b-c88e0f2b0ae6 admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:32:36.346746 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-8c6ed726-889f-4da0-878b-c88e0f2b0ae6 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002143 Jul 11 14:32:36.347254 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 6/10] 10.222.0.72 () {54 vars in 930 bytes} [Mon Jul 11 14:32:36 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:32:36.757768 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-6379f867-338e-4af4-b600-2826a2ff2212 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d2", "swap": 0, "vcpus": 1, "name": "ds1G", "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 1024, "os-flavor-access:is_public": true, "disk": 10, "rxtx_factor": 1.0}} {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:32:36.780014 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-6379f867-338e-4af4-b600-2826a2ff2212 admin admin] 10.222.0.72 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.426408 Jul 11 14:32:36.781024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 5/11] 10.222.0.72 () {64 vars in 1282 bytes} [Mon Jul 11 14:32:36 2022] POST /compute/v2.1/flavors => generated 404 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:32:36.793767 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-0080d998-7ae2-4bbe-bcde-ecb2898f4cff admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:32:36.818891 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-0080d998-7ae2-4bbe-bcde-ecb2898f4cff admin admin] 10.222.0.72 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.029224 Jul 11 14:32:36.819368 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 7/12] 10.222.0.72 () {64 vars in 1335 bytes} [Mon Jul 11 14:32:36 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:32:39.090289 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-6379f867-338e-4af4-b600-2826a2ff2212 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:32:39.091375 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-6379f867-338e-4af4-b600-2826a2ff2212 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001533 Jul 11 14:32:39.091785 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 6/13] 10.222.0.72 () {54 vars in 930 bytes} [Mon Jul 11 14:32:39 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:32:39.551316 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-da91d729-d29a-4405-8a07-24b60df8d55c admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d3", "swap": 0, "name": "ds2G", "vcpus": 2, "rxtx_factor": 1.0, "ram": 2048, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "disk": 10}} {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:32:39.569439 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-da91d729-d29a-4405-8a07-24b60df8d55c admin admin] 10.222.0.72 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.470991 Jul 11 14:32:39.570078 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 8/14] 10.222.0.72 () {64 vars in 1282 bytes} [Mon Jul 11 14:32:39 2022] POST /compute/v2.1/flavors => generated 404 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:32:39.582841 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-7b0a8270-5379-46b8-b09b-b0479ea4bd18 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:32:39.612882 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-7b0a8270-5379-46b8-b09b-b0479ea4bd18 admin admin] 10.222.0.72 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.033720 Jul 11 14:32:39.613471 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 7/15] 10.222.0.72 () {64 vars in 1335 bytes} [Mon Jul 11 14:32:39 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:32:41.887705 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-da91d729-d29a-4405-8a07-24b60df8d55c admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:32:41.888724 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-da91d729-d29a-4405-8a07-24b60df8d55c admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001451 Jul 11 14:32:41.889131 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 9/16] 10.222.0.72 () {54 vars in 930 bytes} [Mon Jul 11 14:32:41 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:32:42.388895 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-4a74b4fe-161e-4476-a338-90f6fd988eda admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "id": "d4", "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 4096, "os-flavor-access:is_public": true, "vcpus": 4, "disk": 20, "name": "ds4G", "swap": 0}} {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:32:42.409434 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-4a74b4fe-161e-4476-a338-90f6fd988eda admin admin] 10.222.0.72 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.514112 Jul 11 14:32:42.410212 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 8/17] 10.222.0.72 () {64 vars in 1282 bytes} [Mon Jul 11 14:32:41 2022] POST /compute/v2.1/flavors => generated 404 bytes in 516 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:32:42.422710 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-ccdb2335-26e5-4fc3-9a7e-13253f387ff6 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:32:42.450360 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-ccdb2335-26e5-4fc3-9a7e-13253f387ff6 admin admin] 10.222.0.72 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.031973 Jul 11 14:32:42.462474 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 10/18] 10.222.0.72 () {64 vars in 1335 bytes} [Mon Jul 11 14:32:42 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:32:45.014996 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-4a74b4fe-161e-4476-a338-90f6fd988eda admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:32:45.016410 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-4a74b4fe-161e-4476-a338-90f6fd988eda admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002008 Jul 11 14:32:45.016959 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 9/19] 10.222.0.72 () {54 vars in 930 bytes} [Mon Jul 11 14:32:45 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:32:45.576753 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-34eb0213-d8e2-43b3-8772-234927a35f76 admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:32:45.603308 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-34eb0213-d8e2-43b3-8772-234927a35f76 admin admin] 10.222.0.72 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.577487 Jul 11 14:32:45.603955 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 11/20] 10.222.0.72 () {60 vars in 1291 bytes} [Mon Jul 11 14:32:45 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 579 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:32:48.254949 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-4a74b4fe-161e-4476-a338-90f6fd988eda admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:32:48.256342 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-4a74b4fe-161e-4476-a338-90f6fd988eda admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002016 Jul 11 14:32:48.256948 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 10/21] 10.222.0.72 () {54 vars in 930 bytes} [Mon Jul 11 14:32:48 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:32:48.812439 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-b9b11718-5452-49ca-98a5-a6e225df8160 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "1", "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.tiny", "vcpus": 1, "ram": 512, "disk": 1, "swap": 0}} {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:32:48.836664 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-b9b11718-5452-49ca-98a5-a6e225df8160 admin admin] 10.222.0.72 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.571592 Jul 11 14:32:48.837389 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 12/22] 10.222.0.72 () {64 vars in 1282 bytes} [Mon Jul 11 14:32:48 2022] POST /compute/v2.1/flavors => generated 402 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:32:48.853575 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-d5c5df98-74ff-493d-bf04-be50f9bc5995 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:32:48.887066 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-d5c5df98-74ff-493d-bf04-be50f9bc5995 admin admin] 10.222.0.72 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.038267 Jul 11 14:32:48.893046 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 11/23] 10.222.0.72 () {64 vars in 1332 bytes} [Mon Jul 11 14:32:48 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:32:51.328420 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-b9b11718-5452-49ca-98a5-a6e225df8160 admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:32:51.331720 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-b9b11718-5452-49ca-98a5-a6e225df8160 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.003685 Jul 11 14:32:51.332759 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 13/24] 10.222.0.72 () {54 vars in 930 bytes} [Mon Jul 11 14:32:51 2022] GET /compute/v2.1 => generated 388 bytes in 6 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:32:51.847293 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-975986a3-cdfe-4f2a-b105-05c4dfdaf669 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.small", "vcpus": 1, "swap": 0, "disk": 20, "id": "2", "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "ram": 2048}} {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:32:51.868553 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-975986a3-cdfe-4f2a-b105-05c4dfdaf669 admin admin] 10.222.0.72 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.527708 Jul 11 14:32:51.869184 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 12/25] 10.222.0.72 () {64 vars in 1282 bytes} [Mon Jul 11 14:32:51 2022] POST /compute/v2.1/flavors => generated 405 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:32:51.882314 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-c953d531-5c01-4117-a3f7-44b20aea2f9a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:32:51.908593 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-c953d531-5c01-4117-a3f7-44b20aea2f9a admin admin] 10.222.0.72 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030586 Jul 11 14:32:51.909178 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 14/26] 10.222.0.72 () {64 vars in 1332 bytes} [Mon Jul 11 14:32:51 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:32:54.262281 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-975986a3-cdfe-4f2a-b105-05c4dfdaf669 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:32:54.263333 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-975986a3-cdfe-4f2a-b105-05c4dfdaf669 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001468 Jul 11 14:32:54.263748 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 13/27] 10.222.0.72 () {54 vars in 930 bytes} [Mon Jul 11 14:32:54 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:32:54.812745 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-373ab265-ed30-47fa-a288-c27bce152211 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "3", "name": "m1.medium", "ram": 4096, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "disk": 40, "swap": 0, "rxtx_factor": 1.0, "vcpus": 2}} {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:32:54.835716 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-373ab265-ed30-47fa-a288-c27bce152211 admin admin] 10.222.0.72 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.565228 Jul 11 14:32:54.836547 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 15/28] 10.222.0.72 () {64 vars in 1282 bytes} [Mon Jul 11 14:32:54 2022] POST /compute/v2.1/flavors => generated 406 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:32:54.849490 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-97ef82f9-ec68-4754-8707-de8f1f208b3b admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:32:54.882861 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-97ef82f9-ec68-4754-8707-de8f1f208b3b admin admin] 10.222.0.72 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.037029 Jul 11 14:32:54.883869 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 14/29] 10.222.0.72 () {64 vars in 1332 bytes} [Mon Jul 11 14:32:54 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:32:57.229032 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-373ab265-ed30-47fa-a288-c27bce152211 admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:32:57.229982 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-373ab265-ed30-47fa-a288-c27bce152211 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001982 Jul 11 14:32:57.230603 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 16/30] 10.222.0.72 () {54 vars in 930 bytes} [Mon Jul 11 14:32:57 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:32:57.714912 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-ddf38173-8820-421a-b5ee-834a170d0a68 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.large", "vcpus": 4, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "ram": 8192, "swap": 0, "id": "4", "disk": 80, "rxtx_factor": 1.0}} {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:32:57.720679 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG dbcounter [-] [105256] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=8,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=105256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:32:57.740698 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-ddf38173-8820-421a-b5ee-834a170d0a68 admin admin] 10.222.0.72 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.504116 Jul 11 14:32:57.741299 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 15/31] 10.222.0.72 () {64 vars in 1282 bytes} [Mon Jul 11 14:32:57 2022] POST /compute/v2.1/flavors => generated 405 bytes in 505 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:32:57.753661 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-6eb7ec93-3c41-431d-afae-f8f65cfad5ce admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:32:57.783219 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-6eb7ec93-3c41-431d-afae-f8f65cfad5ce admin admin] 10.222.0.72 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.033203 Jul 11 14:32:57.796361 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 17/32] 10.222.0.72 () {64 vars in 1332 bytes} [Mon Jul 11 14:32:57 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:33:00.085417 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-ddf38173-8820-421a-b5ee-834a170d0a68 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:00.086601 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-ddf38173-8820-421a-b5ee-834a170d0a68 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001543 Jul 11 14:33:00.087222 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 16/33] 10.222.0.72 () {54 vars in 930 bytes} [Mon Jul 11 14:33:00 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:33:00.693563 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-377e32b4-d13d-4b74-9261-f4a45917c108 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 8, "disk": 160, "name": "m1.xlarge", "swap": 0, "os-flavor-access:is_public": true, "id": "5", "rxtx_factor": 1.0, "ram": 16384, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:33:00.715131 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-377e32b4-d13d-4b74-9261-f4a45917c108 admin admin] 10.222.0.72 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.620674 Jul 11 14:33:00.723061 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 18/34] 10.222.0.72 () {64 vars in 1282 bytes} [Mon Jul 11 14:33:00 2022] POST /compute/v2.1/flavors => generated 408 bytes in 629 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:33:00.729566 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-9a63bb64-c9d2-443b-b872-2b76cfd33d8a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:33:00.771351 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-9a63bb64-c9d2-443b-b872-2b76cfd33d8a admin admin] 10.222.0.72 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.046651 Jul 11 14:33:00.790690 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 17/35] 10.222.0.72 () {64 vars in 1332 bytes} [Mon Jul 11 14:33:00 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:33:21.564299 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-377e32b4-d13d-4b74-9261-f4a45917c108 admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:21.565626 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-377e32b4-d13d-4b74-9261-f4a45917c108 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001799 Jul 11 14:33:21.566131 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 19/36] 10.222.0.72 () {54 vars in 930 bytes} [Mon Jul 11 14:33:21 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:33:21.581720 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG dbcounter [-] [105256] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:RELEASE=1 {{(pid=105256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:33:22.006899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-f0b4f034-51f6-4fbf-8de8-1226dc005bf0 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:22.025631 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-f0b4f034-51f6-4fbf-8de8-1226dc005bf0 admin admin] 10.222.0.72 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.449721 Jul 11 14:33:22.026183 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 18/37] 10.222.0.72 () {60 vars in 1291 bytes} [Mon Jul 11 14:33:21 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:33:23.181654 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-377e32b4-d13d-4b74-9261-f4a45917c108 admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:23.182764 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-377e32b4-d13d-4b74-9261-f4a45917c108 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.004031 Jul 11 14:33:23.183180 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 20/38] 10.222.0.72 () {56 vars in 1074 bytes} [Mon Jul 11 14:33:23 2022] GET /compute/v2.1 => generated 388 bytes in 5 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:33:23.793206 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-4968cf7e-5c39-4eed-924f-edb8a157d883 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:23.799194 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.wsgi [None req-4968cf7e-5c39-4eed-924f-edb8a157d883 admin admin] HTTP exception thrown: Flavor 42 could not be found. Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-4968cf7e-5c39-4eed-924f-edb8a157d883 admin admin] Request method failure captured: Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Accept: application/json Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Accept-Encoding: gzip, deflate Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Connection: keep-alive Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Content-Length: 0 Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Host: 10.222.0.72 Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Openstack-Api-Version: compute 2.90 Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Openstack-System-Scope: None Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: User-Agent: python-novaclient Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Auth-Token: gAAAAABizDSzMGKPn9xPFi2QwyYrX7Jy3wLHhpUygtTBVBRnJ8GxE29n4yUlnP4yfbROQ4JulQC12M1PlG8rmAtG3nhJtFaXEaDXACK6dOTIP4n3EhLC1WVg4JhZOVM4SaoGW87GZMPbO7YmzFntqk8pIxEsP0cMdMJnMpeZIia0Q-k6eDA4Ago Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Domain-Id: None Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Domain-Name: None Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Identity-Status: Confirmed Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Is-Admin-Project: True Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Openstack-Nova-Api-Version: 2.90 Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Project-Domain-Id: default Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Project-Domain-Name: Default Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Project-Id: e2b4fae2f8fe4791bd2046666efe9390 Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Project-Name: admin Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Role: member,reader,admin Jul 11 14:33:23.803024 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Roles: member,reader,admin Jul 11 14:33:23.804802 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/e2b4fae2f8fe4791bd2046666efe9390"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/e2b4fae2f8fe4791bd2046666efe9390"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_e2b4fae2f8fe4791bd2046666efe9390"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/e2b4fae2f8fe4791bd2046666efe9390"}]}] Jul 11 14:33:23.804802 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Tenant: admin Jul 11 14:33:23.804802 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Tenant-Id: e2b4fae2f8fe4791bd2046666efe9390 Jul 11 14:33:23.804802 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Tenant-Name: admin Jul 11 14:33:23.804802 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-User: admin Jul 11 14:33:23.804802 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-User-Domain-Id: default Jul 11 14:33:23.804802 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-User-Domain-Name: Default Jul 11 14:33:23.804802 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-User-Id: a7d8b4db44114e1ba544d78dc12ef30a Jul 11 14:33:23.804802 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-User-Name: admin Jul 11 14:33:23.804802 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: method: > Jul 11 14:33:23.804802 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: exception: Flavor 42 could not be found. Jul 11 14:33:23.804802 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: action_args: {'id': '42'} Jul 11 14:33:23.804802 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:33:23.804802 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:33:23.806456 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:33:23.807899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:33:23.807899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:33:23.807899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:33:23.807899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:33:23.807899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 11 14:33:23.807899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:33:23.807899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Jul 11 14:33:23.807899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Jul 11 14:33:23.807899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:33:23.807899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Jul 11 14:33:23.807899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:33:23.807899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:33:23.807899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:33:23.807899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:33:23.807899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:33:23.807899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Jul 11 14:33:23.807899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Jul 11 14:33:23.807899 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-4968cf7e-5c39-4eed-924f-edb8a157d883 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105256) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:33:23.809336 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-4968cf7e-5c39-4eed-924f-edb8a157d883 admin admin] 10.222.0.72 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.468181 Jul 11 14:33:23.809336 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 19/39] 10.222.0.72 () {60 vars in 1190 bytes} [Mon Jul 11 14:33:23 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 469 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 11 14:33:23.815265 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-24b6c692-ceaf-430e-9d9d-4013f4fb33f9 admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:23.818530 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG dbcounter [-] [105257] Writing DB stats nova_api:SELECT=47,nova_api:SAVEPOINT=5,nova_api:INSERT=10,nova_api:RELEASE=5 {{(pid=105257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:33:23.840124 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-24b6c692-ceaf-430e-9d9d-4013f4fb33f9 admin admin] 10.222.0.72 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.028619 Jul 11 14:33:23.840749 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 21/40] 10.222.0.72 () {60 vars in 1210 bytes} [Mon Jul 11 14:33:23 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:33:23.850794 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-f6fb18bf-14d3-415c-b803-7dc702b00ea2 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:23.872960 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-f6fb18bf-14d3-415c-b803-7dc702b00ea2 admin admin] 10.222.0.72 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.025767 Jul 11 14:33:23.873547 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 20/41] 10.222.0.72 () {60 vars in 1210 bytes} [Mon Jul 11 14:33:23 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:33:23.891557 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-d05076f1-8ef2-4cd6-b560-a4167489e2cf admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:23.898422 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.wsgi [None req-d05076f1-8ef2-4cd6-b560-a4167489e2cf admin admin] HTTP exception thrown: Flavor 42 could not be found. Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-d05076f1-8ef2-4cd6-b560-a4167489e2cf admin admin] Request method failure captured: Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Accept: application/json Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Accept-Encoding: gzip, deflate Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Connection: keep-alive Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Content-Length: 0 Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Host: 10.222.0.72 Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Openstack-Api-Version: compute 2.90 Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Openstack-System-Scope: None Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: User-Agent: python-novaclient Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Auth-Token: gAAAAABizDSzMGKPn9xPFi2QwyYrX7Jy3wLHhpUygtTBVBRnJ8GxE29n4yUlnP4yfbROQ4JulQC12M1PlG8rmAtG3nhJtFaXEaDXACK6dOTIP4n3EhLC1WVg4JhZOVM4SaoGW87GZMPbO7YmzFntqk8pIxEsP0cMdMJnMpeZIia0Q-k6eDA4Ago Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Domain-Id: None Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Domain-Name: None Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Identity-Status: Confirmed Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Is-Admin-Project: True Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Openstack-Nova-Api-Version: 2.90 Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Project-Domain-Id: default Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Project-Domain-Name: Default Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Project-Id: e2b4fae2f8fe4791bd2046666efe9390 Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Project-Name: admin Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Role: member,reader,admin Jul 11 14:33:23.901818 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Roles: member,reader,admin Jul 11 14:33:23.903519 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/e2b4fae2f8fe4791bd2046666efe9390"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/e2b4fae2f8fe4791bd2046666efe9390"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_e2b4fae2f8fe4791bd2046666efe9390"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/e2b4fae2f8fe4791bd2046666efe9390"}]}] Jul 11 14:33:23.903519 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Tenant: admin Jul 11 14:33:23.903519 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Tenant-Id: e2b4fae2f8fe4791bd2046666efe9390 Jul 11 14:33:23.903519 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Tenant-Name: admin Jul 11 14:33:23.903519 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-User: admin Jul 11 14:33:23.903519 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-User-Domain-Id: default Jul 11 14:33:23.903519 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-User-Domain-Name: Default Jul 11 14:33:23.903519 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-User-Id: a7d8b4db44114e1ba544d78dc12ef30a Jul 11 14:33:23.903519 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-User-Name: admin Jul 11 14:33:23.903519 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: method: > Jul 11 14:33:23.903519 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: exception: Flavor 42 could not be found. Jul 11 14:33:23.903519 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: action_args: {'id': '42'} Jul 11 14:33:23.903519 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:33:23.903519 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:33:23.904943 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:33:23.906462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:33:23.906462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:33:23.906462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:33:23.906462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:33:23.906462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 11 14:33:23.906462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:33:23.906462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Jul 11 14:33:23.906462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Jul 11 14:33:23.906462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:33:23.906462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Jul 11 14:33:23.906462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:33:23.906462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:33:23.906462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:33:23.906462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:33:23.906462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:33:23.906462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Jul 11 14:33:23.906462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Jul 11 14:33:23.906462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-d05076f1-8ef2-4cd6-b560-a4167489e2cf admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105257) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:33:23.907782 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-d05076f1-8ef2-4cd6-b560-a4167489e2cf admin admin] 10.222.0.72 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.90 time: 0.015809 Jul 11 14:33:23.907782 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 22/42] 10.222.0.72 () {60 vars in 1190 bytes} [Mon Jul 11 14:33:23 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 17 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Jul 11 14:33:25.288771 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-f6fb18bf-14d3-415c-b803-7dc702b00ea2 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:25.290600 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-f6fb18bf-14d3-415c-b803-7dc702b00ea2 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002488 Jul 11 14:33:25.291343 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 21/43] 10.222.0.72 () {56 vars in 1074 bytes} [Mon Jul 11 14:33:25 2022] GET /compute/v2.1 => generated 388 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:33:25.961319 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-8e224cd9-87fd-492b-9227-9c52f336fbf1 admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:25.968880 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.wsgi [None req-8e224cd9-87fd-492b-9227-9c52f336fbf1 admin admin] HTTP exception thrown: Flavor 84 could not be found. Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-8e224cd9-87fd-492b-9227-9c52f336fbf1 admin admin] Request method failure captured: Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Accept: application/json Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Accept-Encoding: gzip, deflate Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Connection: keep-alive Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Content-Length: 0 Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Host: 10.222.0.72 Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Openstack-Api-Version: compute 2.90 Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Openstack-System-Scope: None Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: User-Agent: python-novaclient Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Auth-Token: gAAAAABizDS1CHqDU5Vpi0Krz4j4RGehENM3J3i_OEO0MJhOQ1gAzwURSTr2TJCNlaRO1Lk9HWfW44hWYz1a-dE3BHOeZTQ7z93GfqyX0OPLI-7rKQVkfu7H8w-AuJQcLbF9PBGnmbZYIeia4CrBWPhnFb2T965ktKaML5iuvP-m5MnQcVVJzKw Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Domain-Id: None Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Domain-Name: None Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Identity-Status: Confirmed Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Is-Admin-Project: True Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Openstack-Nova-Api-Version: 2.90 Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Project-Domain-Id: default Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Project-Domain-Name: Default Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Project-Id: e2b4fae2f8fe4791bd2046666efe9390 Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Project-Name: admin Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Role: member,reader,admin Jul 11 14:33:25.970903 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Roles: member,reader,admin Jul 11 14:33:25.972462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/e2b4fae2f8fe4791bd2046666efe9390"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/e2b4fae2f8fe4791bd2046666efe9390"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_e2b4fae2f8fe4791bd2046666efe9390"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/e2b4fae2f8fe4791bd2046666efe9390"}]}] Jul 11 14:33:25.972462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Tenant: admin Jul 11 14:33:25.972462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Tenant-Id: e2b4fae2f8fe4791bd2046666efe9390 Jul 11 14:33:25.972462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Tenant-Name: admin Jul 11 14:33:25.972462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-User: admin Jul 11 14:33:25.972462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-User-Domain-Id: default Jul 11 14:33:25.972462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-User-Domain-Name: Default Jul 11 14:33:25.972462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-User-Id: a7d8b4db44114e1ba544d78dc12ef30a Jul 11 14:33:25.972462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-User-Name: admin Jul 11 14:33:25.972462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: method: > Jul 11 14:33:25.972462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: exception: Flavor 84 could not be found. Jul 11 14:33:25.972462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: action_args: {'id': '84'} Jul 11 14:33:25.972462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:33:25.972462 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:33:25.973295 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:33:25.974594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:33:25.974594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:33:25.974594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:33:25.974594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:33:25.974594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 11 14:33:25.974594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:33:25.974594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Jul 11 14:33:25.974594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Jul 11 14:33:25.974594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:33:25.974594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Jul 11 14:33:25.974594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:33:25.974594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:33:25.974594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:33:25.974594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:33:25.974594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:33:25.974594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Jul 11 14:33:25.974594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Jul 11 14:33:25.974594 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-8e224cd9-87fd-492b-9227-9c52f336fbf1 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105257) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:33:25.976776 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-8e224cd9-87fd-492b-9227-9c52f336fbf1 admin admin] 10.222.0.72 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.522657 Jul 11 14:33:25.976776 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 23/44] 10.222.0.72 () {60 vars in 1190 bytes} [Mon Jul 11 14:33:25 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 524 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 11 14:33:25.994015 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-d0eb47ae-8edc-4a76-8295-e21bc4a9af2a admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:26.018259 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-d0eb47ae-8edc-4a76-8295-e21bc4a9af2a admin admin] 10.222.0.72 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.029851 Jul 11 14:33:26.019190 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 22/45] 10.222.0.72 () {60 vars in 1210 bytes} [Mon Jul 11 14:33:25 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:33:26.032461 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-d9cf8de4-0e13-4f95-a75a-1d41701498a1 admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:26.047429 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-d9cf8de4-0e13-4f95-a75a-1d41701498a1 admin admin] 10.222.0.72 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.018682 Jul 11 14:33:26.048004 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 24/46] 10.222.0.72 () {60 vars in 1210 bytes} [Mon Jul 11 14:33:26 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:33:26.058651 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-e45680a4-f4aa-4ee6-bb3d-85d48f744113 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:26.064487 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.wsgi [None req-e45680a4-f4aa-4ee6-bb3d-85d48f744113 admin admin] HTTP exception thrown: Flavor 84 could not be found. Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-e45680a4-f4aa-4ee6-bb3d-85d48f744113 admin admin] Request method failure captured: Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Accept: application/json Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Accept-Encoding: gzip, deflate Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Connection: keep-alive Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Content-Length: 0 Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Host: 10.222.0.72 Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Openstack-Api-Version: compute 2.90 Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Openstack-System-Scope: None Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: User-Agent: python-novaclient Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Auth-Token: gAAAAABizDS1CHqDU5Vpi0Krz4j4RGehENM3J3i_OEO0MJhOQ1gAzwURSTr2TJCNlaRO1Lk9HWfW44hWYz1a-dE3BHOeZTQ7z93GfqyX0OPLI-7rKQVkfu7H8w-AuJQcLbF9PBGnmbZYIeia4CrBWPhnFb2T965ktKaML5iuvP-m5MnQcVVJzKw Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Domain-Id: None Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Domain-Name: None Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Identity-Status: Confirmed Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Is-Admin-Project: True Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Openstack-Nova-Api-Version: 2.90 Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Project-Domain-Id: default Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Project-Domain-Name: Default Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Project-Id: e2b4fae2f8fe4791bd2046666efe9390 Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Project-Name: admin Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Role: member,reader,admin Jul 11 14:33:26.066070 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Roles: member,reader,admin Jul 11 14:33:26.067681 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/e2b4fae2f8fe4791bd2046666efe9390"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/e2b4fae2f8fe4791bd2046666efe9390"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_e2b4fae2f8fe4791bd2046666efe9390"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/e2b4fae2f8fe4791bd2046666efe9390"}]}] Jul 11 14:33:26.067681 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Tenant: admin Jul 11 14:33:26.067681 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Tenant-Id: e2b4fae2f8fe4791bd2046666efe9390 Jul 11 14:33:26.067681 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Tenant-Name: admin Jul 11 14:33:26.067681 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-User: admin Jul 11 14:33:26.067681 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-User-Domain-Id: default Jul 11 14:33:26.067681 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-User-Domain-Name: Default Jul 11 14:33:26.067681 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-User-Id: a7d8b4db44114e1ba544d78dc12ef30a Jul 11 14:33:26.067681 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-User-Name: admin Jul 11 14:33:26.067681 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: method: > Jul 11 14:33:26.067681 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: exception: Flavor 84 could not be found. Jul 11 14:33:26.067681 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: action_args: {'id': '84'} Jul 11 14:33:26.067681 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:33:26.067681 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:33:26.068615 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:33:26.070032 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:33:26.070032 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:33:26.070032 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:33:26.070032 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:33:26.070032 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 11 14:33:26.070032 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:33:26.070032 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Jul 11 14:33:26.070032 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Jul 11 14:33:26.070032 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:33:26.070032 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Jul 11 14:33:26.070032 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:33:26.070032 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:33:26.070032 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:33:26.070032 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:33:26.070032 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:33:26.070032 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Jul 11 14:33:26.070032 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Jul 11 14:33:26.070032 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-e45680a4-f4aa-4ee6-bb3d-85d48f744113 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105256) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:33:26.071310 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-e45680a4-f4aa-4ee6-bb3d-85d48f744113 admin admin] 10.222.0.72 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.90 time: 0.013145 Jul 11 14:33:26.071310 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 23/47] 10.222.0.72 () {60 vars in 1190 bytes} [Mon Jul 11 14:33:26 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 14 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 11 14:33:27.165898 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-d9cf8de4-0e13-4f95-a75a-1d41701498a1 admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:27.167146 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-d9cf8de4-0e13-4f95-a75a-1d41701498a1 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002277 Jul 11 14:33:27.167574 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 25/48] 10.222.0.72 () {56 vars in 1074 bytes} [Mon Jul 11 14:33:27 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:33:27.875338 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-b1fd428c-83bb-480d-8142-c0cf8fdc7c4c admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:27.883177 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.wsgi [None req-b1fd428c-83bb-480d-8142-c0cf8fdc7c4c admin admin] HTTP exception thrown: Flavor 451 could not be found. Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-b1fd428c-83bb-480d-8142-c0cf8fdc7c4c admin admin] Request method failure captured: Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Accept: application/json Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Accept-Encoding: gzip, deflate Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Connection: keep-alive Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Content-Length: 0 Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Host: 10.222.0.72 Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Openstack-Api-Version: compute 2.90 Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Openstack-System-Scope: None Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: User-Agent: python-novaclient Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Auth-Token: gAAAAABizDS3tad4Cxqw2g_dYAFUlYNpa8lCeLSNswiT8bo38uuDhnpSLbyyHtIZiZPRAH-Et8ia_jvgbj5w5yiulXwf0afWXqap6E0ShOVJ-P1Mkxa6uqmIjGM44m6md2nDEOlujYKPs7_c5YbGMdkdmMzD1uIPEjyTjqCBP6442z7qagRHxHM Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Domain-Id: None Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Domain-Name: None Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Identity-Status: Confirmed Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Is-Admin-Project: True Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Openstack-Nova-Api-Version: 2.90 Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Project-Domain-Id: default Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Project-Domain-Name: Default Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Project-Id: e2b4fae2f8fe4791bd2046666efe9390 Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Project-Name: admin Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Role: member,reader,admin Jul 11 14:33:27.884922 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Roles: member,reader,admin Jul 11 14:33:27.887605 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/e2b4fae2f8fe4791bd2046666efe9390"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/e2b4fae2f8fe4791bd2046666efe9390"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_e2b4fae2f8fe4791bd2046666efe9390"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/e2b4fae2f8fe4791bd2046666efe9390"}]}] Jul 11 14:33:27.887605 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Tenant: admin Jul 11 14:33:27.887605 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Tenant-Id: e2b4fae2f8fe4791bd2046666efe9390 Jul 11 14:33:27.887605 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-Tenant-Name: admin Jul 11 14:33:27.887605 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-User: admin Jul 11 14:33:27.887605 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-User-Domain-Id: default Jul 11 14:33:27.887605 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-User-Domain-Name: Default Jul 11 14:33:27.887605 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-User-Id: a7d8b4db44114e1ba544d78dc12ef30a Jul 11 14:33:27.887605 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: X-User-Name: admin Jul 11 14:33:27.887605 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: method: > Jul 11 14:33:27.887605 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: exception: Flavor 451 could not be found. Jul 11 14:33:27.887605 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: action_args: {'id': '451'} Jul 11 14:33:27.887605 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:33:27.887605 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:33:27.889245 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:33:27.891330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:33:27.891330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:33:27.891330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:33:27.891330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:33:27.891330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 11 14:33:27.891330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:33:27.891330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Jul 11 14:33:27.891330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Jul 11 14:33:27.891330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:33:27.891330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Jul 11 14:33:27.891330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:33:27.891330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:33:27.891330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:33:27.891330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:33:27.891330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:33:27.891330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Jul 11 14:33:27.891330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: ERROR nova.api.openstack.wsgi Jul 11 14:33:27.891330 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-b1fd428c-83bb-480d-8142-c0cf8fdc7c4c admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105256) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:33:27.893551 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-b1fd428c-83bb-480d-8142-c0cf8fdc7c4c admin admin] 10.222.0.72 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.497604 Jul 11 14:33:27.893551 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 24/49] 10.222.0.72 () {60 vars in 1193 bytes} [Mon Jul 11 14:33:27 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 499 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 11 14:33:27.898447 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-b361d4e7-0627-4ea3-90ca-0435203a0631 admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:27.918297 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-b361d4e7-0627-4ea3-90ca-0435203a0631 admin admin] 10.222.0.72 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.024637 Jul 11 14:33:27.918991 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 26/50] 10.222.0.72 () {60 vars in 1210 bytes} [Mon Jul 11 14:33:27 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 26 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:33:27.931458 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-630a1a18-4f68-4d9e-83e1-406ae9ad27c6 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:27.947569 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-630a1a18-4f68-4d9e-83e1-406ae9ad27c6 admin admin] 10.222.0.72 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.90 time: 0.020961 Jul 11 14:33:27.948149 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 25/51] 10.222.0.72 () {60 vars in 1210 bytes} [Mon Jul 11 14:33:27 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:33:27.961537 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-8b839900-2552-49af-bed1-b1869620c71e admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:27.968177 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.wsgi [None req-8b839900-2552-49af-bed1-b1869620c71e admin admin] HTTP exception thrown: Flavor 451 could not be found. Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-8b839900-2552-49af-bed1-b1869620c71e admin admin] Request method failure captured: Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Accept: application/json Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Accept-Encoding: gzip, deflate Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Connection: keep-alive Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Content-Length: 0 Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Host: 10.222.0.72 Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Openstack-Api-Version: compute 2.90 Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Openstack-System-Scope: None Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: User-Agent: python-novaclient Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Auth-Token: gAAAAABizDS3tad4Cxqw2g_dYAFUlYNpa8lCeLSNswiT8bo38uuDhnpSLbyyHtIZiZPRAH-Et8ia_jvgbj5w5yiulXwf0afWXqap6E0ShOVJ-P1Mkxa6uqmIjGM44m6md2nDEOlujYKPs7_c5YbGMdkdmMzD1uIPEjyTjqCBP6442z7qagRHxHM Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Domain-Id: None Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Domain-Name: None Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Identity-Status: Confirmed Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Is-Admin-Project: True Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Openstack-Nova-Api-Version: 2.90 Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Project-Domain-Id: default Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Project-Domain-Name: Default Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Project-Id: e2b4fae2f8fe4791bd2046666efe9390 Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Project-Name: admin Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Role: member,reader,admin Jul 11 14:33:27.969799 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Roles: member,reader,admin Jul 11 14:33:27.971780 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/e2b4fae2f8fe4791bd2046666efe9390"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/e2b4fae2f8fe4791bd2046666efe9390"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_e2b4fae2f8fe4791bd2046666efe9390"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/e2b4fae2f8fe4791bd2046666efe9390"}]}] Jul 11 14:33:27.971780 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Tenant: admin Jul 11 14:33:27.971780 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Tenant-Id: e2b4fae2f8fe4791bd2046666efe9390 Jul 11 14:33:27.971780 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-Tenant-Name: admin Jul 11 14:33:27.971780 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-User: admin Jul 11 14:33:27.971780 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-User-Domain-Id: default Jul 11 14:33:27.971780 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-User-Domain-Name: Default Jul 11 14:33:27.971780 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-User-Id: a7d8b4db44114e1ba544d78dc12ef30a Jul 11 14:33:27.971780 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: X-User-Name: admin Jul 11 14:33:27.971780 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: method: > Jul 11 14:33:27.971780 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: exception: Flavor 451 could not be found. Jul 11 14:33:27.971780 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: action_args: {'id': '451'} Jul 11 14:33:27.971780 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:33:27.971780 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:33:27.973116 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:33:27.975227 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:33:27.975227 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:33:27.975227 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:33:27.975227 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:33:27.975227 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Jul 11 14:33:27.975227 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:33:27.975227 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Jul 11 14:33:27.975227 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Jul 11 14:33:27.975227 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:33:27.975227 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Jul 11 14:33:27.975227 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:33:27.975227 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:33:27.975227 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:33:27.975227 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:33:27.975227 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:33:27.975227 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Jul 11 14:33:27.975227 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: ERROR nova.api.openstack.wsgi Jul 11 14:33:27.975227 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-8b839900-2552-49af-bed1-b1869620c71e admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105257) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:33:27.976958 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-8b839900-2552-49af-bed1-b1869620c71e admin admin] 10.222.0.72 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.90 time: 0.015423 Jul 11 14:33:27.976958 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 27/52] 10.222.0.72 () {60 vars in 1193 bytes} [Mon Jul 11 14:33:27 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 17 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Jul 11 14:33:29.071184 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-630a1a18-4f68-4d9e-83e1-406ae9ad27c6 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:29.073115 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-630a1a18-4f68-4d9e-83e1-406ae9ad27c6 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002410 Jul 11 14:33:29.073673 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 26/53] 10.222.0.72 () {56 vars in 1074 bytes} [Mon Jul 11 14:33:29 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:33:29.739169 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-d8ce745a-9879-4abf-a142-2234b56b3225 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:33:29.764294 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-d8ce745a-9879-4abf-a142-2234b56b3225 admin admin] 10.222.0.72 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.90 time: 0.546717 Jul 11 14:33:29.765338 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 28/54] 10.222.0.72 () {64 vars in 1235 bytes} [Mon Jul 11 14:33:29 2022] POST /compute/v2.1/flavors => generated 404 bytes in 549 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 11 14:33:30.825256 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-630a1a18-4f68-4d9e-83e1-406ae9ad27c6 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:30.827022 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-630a1a18-4f68-4d9e-83e1-406ae9ad27c6 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002424 Jul 11 14:33:30.827826 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 27/55] 10.222.0.72 () {56 vars in 1074 bytes} [Mon Jul 11 14:33:30 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:33:31.433052 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-d3244b7d-445b-47d0-8835-2849c52e97bb admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:33:31.453160 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-d3244b7d-445b-47d0-8835-2849c52e97bb admin admin] 10.222.0.72 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.90 time: 0.473999 Jul 11 14:33:31.453757 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 29/56] 10.222.0.72 () {64 vars in 1235 bytes} [Mon Jul 11 14:33:30 2022] POST /compute/v2.1/flavors => generated 405 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:33:32.523926 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-630a1a18-4f68-4d9e-83e1-406ae9ad27c6 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:32.526052 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-630a1a18-4f68-4d9e-83e1-406ae9ad27c6 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001945 Jul 11 14:33:32.526744 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 28/57] 10.222.0.72 () {56 vars in 1074 bytes} [Mon Jul 11 14:33:32 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:33:33.156850 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-a15204dc-e724-4dc1-a78e-416f3387fc35 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:33:33.190228 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-a15204dc-e724-4dc1-a78e-416f3387fc35 admin admin] 10.222.0.72 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.90 time: 0.505345 Jul 11 14:33:33.190815 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 30/58] 10.222.0.72 () {64 vars in 1235 bytes} [Mon Jul 11 14:33:32 2022] POST /compute/v2.1/flavors => generated 407 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:33:34.221215 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-630a1a18-4f68-4d9e-83e1-406ae9ad27c6 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:34.222943 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-630a1a18-4f68-4d9e-83e1-406ae9ad27c6 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002326 Jul 11 14:33:34.223663 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 29/59] 10.222.0.72 () {56 vars in 1074 bytes} [Mon Jul 11 14:33:34 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:33:34.898128 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-dc54b0d4-5b31-4e2a-91b9-dfeb8039de73 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:33:34.919306 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-dc54b0d4-5b31-4e2a-91b9-dfeb8039de73 admin admin] 10.222.0.72 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.90 time: 0.494858 Jul 11 14:33:34.919865 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 31/60] 10.222.0.72 () {64 vars in 1235 bytes} [Mon Jul 11 14:33:34 2022] POST /compute/v2.1/flavors => generated 409 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 11 14:33:35.989778 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-630a1a18-4f68-4d9e-83e1-406ae9ad27c6 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:35.991254 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-630a1a18-4f68-4d9e-83e1-406ae9ad27c6 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002630 Jul 11 14:33:35.991669 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 30/61] 10.222.0.72 () {56 vars in 1074 bytes} [Mon Jul 11 14:33:35 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:33:36.627742 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-cc0a25df-6799-405e-860e-db552a329f55 admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:33:36.648509 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-cc0a25df-6799-405e-860e-db552a329f55 admin admin] 10.222.0.72 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.90 time: 0.505740 Jul 11 14:33:36.649084 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 32/62] 10.222.0.72 () {60 vars in 1202 bytes} [Mon Jul 11 14:33:36 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:34:33.079345 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-630a1a18-4f68-4d9e-83e1-406ae9ad27c6 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:34:33.081057 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-630a1a18-4f68-4d9e-83e1-406ae9ad27c6 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002437 Jul 11 14:34:33.081757 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 31/63] 10.222.0.72 () {54 vars in 930 bytes} [Mon Jul 11 14:34:33 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:34:33.100218 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG dbcounter [-] [105257] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=105257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:34:33.596741 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-ed94bcf9-6201-4997-99ca-fca0a038b6f5 admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:34:33.616707 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-ed94bcf9-6201-4997-99ca-fca0a038b6f5 admin admin] 10.222.0.72 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.524338 Jul 11 14:34:33.617269 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 33/64] 10.222.0.72 () {60 vars in 1291 bytes} [Mon Jul 11 14:34:33 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:34:33.644587 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-d8b46b22-184b-40af-8423-bdc8d72264ba admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:34:33.651046 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG dbcounter [-] [105256] Writing DB stats nova_api:SELECT=14 {{(pid=105256) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:34:33.658621 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-d8b46b22-184b-40af-8423-bdc8d72264ba admin admin] 10.222.0.72 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.020894 Jul 11 14:34:33.659199 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 32/65] 10.222.0.72 () {60 vars in 1282 bytes} [Mon Jul 11 14:34:33 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:34:33.673691 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-eb360dfc-c4f9-428e-9e5d-231fed70744f admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:34:33.682125 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-eb360dfc-c4f9-428e-9e5d-231fed70744f admin admin] 10.222.0.72 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012086 Jul 11 14:34:33.682916 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 34/66] 10.222.0.72 () {60 vars in 1285 bytes} [Mon Jul 11 14:34:33 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:34:33.695210 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-2f7d58f7-0496-453a-9d48-ce10809214f1 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:34:33.704092 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-2f7d58f7-0496-453a-9d48-ce10809214f1 admin admin] 10.222.0.72 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013131 Jul 11 14:34:33.704673 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 33/67] 10.222.0.72 () {60 vars in 1285 bytes} [Mon Jul 11 14:34:33 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:34:33.721242 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-cf50dcb1-f293-4e6d-926e-f2362b579960 admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:34:33.729777 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-cf50dcb1-f293-4e6d-926e-f2362b579960 admin admin] 10.222.0.72 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012205 Jul 11 14:34:33.730454 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 35/68] 10.222.0.72 () {60 vars in 1282 bytes} [Mon Jul 11 14:34:33 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:38:09.358242 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-2f7d58f7-0496-453a-9d48-ce10809214f1 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:38:09.359361 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-2f7d58f7-0496-453a-9d48-ce10809214f1 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001556 Jul 11 14:38:09.360473 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 34/69] 10.222.0.72 () {56 vars in 1074 bytes} [Mon Jul 11 14:38:09 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:38:09.506509 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG dbcounter [-] [105257] Writing DB stats nova_api:SELECT=6 {{(pid=105257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:38:09.672630 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-03266af9-d27c-4fde-a0ca-b763661c4c85 admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:38:09.674254 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:38:09.674254 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: warnings.warn(msg) Jul 11 14:38:09.674999 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-03266af9-d27c-4fde-a0ca-b763661c4c85 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:38:09.675444 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-03266af9-d27c-4fde-a0ca-b763661c4c85 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:38:09.676238 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-03266af9-d27c-4fde-a0ca-b763661c4c85 admin admin] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:38:09.676663 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-03266af9-d27c-4fde-a0ca-b763661c4c85 admin admin] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:38:09.712098 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-03266af9-d27c-4fde-a0ca-b763661c4c85 admin admin] 10.222.0.72 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.206941 Jul 11 14:38:09.712670 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 36/70] 10.222.0.72 () {60 vars in 1193 bytes} [Mon Jul 11 14:38:09 2022] GET /compute/v2.1/os-services => generated 791 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:38:10.560628 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-2f7d58f7-0496-453a-9d48-ce10809214f1 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:38:10.561691 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-2f7d58f7-0496-453a-9d48-ce10809214f1 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001697 Jul 11 14:38:10.562103 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 35/71] 10.222.0.72 () {56 vars in 1074 bytes} [Mon Jul 11 14:38:10 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:38:10.771354 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-bc68d460-77e0-457e-96f0-d9885756a619 admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:38:10.772552 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:38:10.772552 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: warnings.warn(msg) Jul 11 14:38:10.773754 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-bc68d460-77e0-457e-96f0-d9885756a619 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:38:10.774192 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-bc68d460-77e0-457e-96f0-d9885756a619 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:38:10.775022 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-bc68d460-77e0-457e-96f0-d9885756a619 admin admin] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:38:10.775537 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-bc68d460-77e0-457e-96f0-d9885756a619 admin admin] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:38:10.796305 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-bc68d460-77e0-457e-96f0-d9885756a619 admin admin] 10.222.0.72 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.90 time: 0.085622 Jul 11 14:38:10.796896 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 37/72] 10.222.0.72 () {60 vars in 1193 bytes} [Mon Jul 11 14:38:10 2022] GET /compute/v2.1/os-services => generated 791 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:38:22.083717 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-2f7d58f7-0496-453a-9d48-ce10809214f1 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:38:22.084713 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-2f7d58f7-0496-453a-9d48-ce10809214f1 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001409 Jul 11 14:38:22.085107 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 36/73] 10.222.0.72 () {56 vars in 1074 bytes} [Mon Jul 11 14:38:22 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:38:22.229598 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG dbcounter [-] [105257] Writing DB stats nova_cell0:SELECT=4 {{(pid=105257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:38:22.230526 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG dbcounter [-] [105257] Writing DB stats nova_cell1:SELECT=4 {{(pid=105257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:38:22.231322 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG dbcounter [-] [105257] Writing DB stats nova_api:SELECT=8 {{(pid=105257) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:38:22.283454 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-c2a95a5b-62b8-4c96-821d-430734e1e1e8 admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:38:22.284624 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:38:22.284624 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: warnings.warn(msg) Jul 11 14:38:22.285809 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-c2a95a5b-62b8-4c96-821d-430734e1e1e8 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:38:22.286289 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-c2a95a5b-62b8-4c96-821d-430734e1e1e8 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:38:22.287130 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-c2a95a5b-62b8-4c96-821d-430734e1e1e8 admin admin] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:38:22.287561 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-c2a95a5b-62b8-4c96-821d-430734e1e1e8 admin admin] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:38:22.313154 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-c2a95a5b-62b8-4c96-821d-430734e1e1e8 admin admin] 10.222.0.72 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.90 time: 0.088061 Jul 11 14:38:22.313643 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 38/74] 10.222.0.72 () {60 vars in 1193 bytes} [Mon Jul 11 14:38:22 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:38:23.159966 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: DEBUG nova.api.openstack.wsgi [None req-2f7d58f7-0496-453a-9d48-ce10809214f1 admin admin] Calling method '>' {{(pid=105256) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:38:23.160972 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: INFO nova.api.openstack.requestlog [None req-2f7d58f7-0496-453a-9d48-ce10809214f1 admin admin] 10.222.0.72 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001432 Jul 11 14:38:23.161375 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: [pid: 105256|app: 0|req: 37/75] 10.222.0.72 () {56 vars in 1074 bytes} [Mon Jul 11 14:38:23 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:38:23.363077 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG nova.api.openstack.wsgi [None req-5e51316f-fac6-40c3-8be5-f26e1121abf7 admin admin] Calling method '>' {{(pid=105257) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:38:23.364291 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "os_compute_api:os-services:list": "rule:context_is_admin" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:38:23.364291 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: warnings.warn(msg) Jul 11 14:38:23.365483 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-5e51316f-fac6-40c3-8be5-f26e1121abf7 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:38:23.365977 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-5e51316f-fac6-40c3-8be5-f26e1121abf7 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:38:23.366746 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-5e51316f-fac6-40c3-8be5-f26e1121abf7 admin admin] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:38:23.367183 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: DEBUG oslo_concurrency.lockutils [None req-5e51316f-fac6-40c3-8be5-f26e1121abf7 admin admin] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105257) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:38:23.390145 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: INFO nova.api.openstack.requestlog [None req-5e51316f-fac6-40c3-8be5-f26e1121abf7 admin admin] 10.222.0.72 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.90 time: 0.080604 Jul 11 14:38:23.391268 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: [pid: 105257|app: 0|req: 39/76] 10.222.0.72 () {60 vars in 1193 bytes} [Mon Jul 11 14:38:23 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:39:21.037633 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: Mon Jul 11 14:39:21 2022 - graceful shutdown triggered... Jul 11 14:39:21.037730 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Jul 11 14:39:21.038873 dv-cinder-master-3-iscsi devstack@n-api.service[105256]: Gracefully killing worker 1 (pid: 105256)... Jul 11 14:39:21.039473 dv-cinder-master-3-iscsi devstack@n-api.service[105257]: Gracefully killing worker 2 (pid: 105257)... Jul 11 14:39:22.039937 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: worker 1 buried after 1 seconds Jul 11 14:39:22.039937 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: worker 2 buried after 1 seconds Jul 11 14:39:22.039937 dv-cinder-master-3-iscsi devstack@n-api.service[105255]: goodbye to uWSGI. Jul 11 14:39:22.041579 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Jul 11 14:39:22.042129 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Jul 11 14:39:22.045528 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Jul 11 14:39:22.055645 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Jul 11 14:39:22.057588 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 11 14:39:22 2022] *** Jul 11 14:39:22.057588 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 11 14:39:22.057588 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 11 14:39:22.057588 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: nodename: dv-cinder-master-3-iscsi Jul 11 14:39:22.057588 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: machine: x86_64 Jul 11 14:39:22.057588 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: clock source: unix Jul 11 14:39:22.057588 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: pcre jit disabled Jul 11 14:39:22.057588 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: detected number of CPU cores: 4 Jul 11 14:39:22.057588 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: current working directory: / Jul 11 14:39:22.057588 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: detected binary path: /usr/bin/uwsgi-core Jul 11 14:39:22.058933 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: your processes number limit is 47680 Jul 11 14:39:22.058933 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: your memory page size is 4096 bytes Jul 11 14:39:22.058933 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: detected max file descriptor number: 2048 Jul 11 14:39:22.058933 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: lock engine: pthread robust mutexes Jul 11 14:39:22.058933 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: thunder lock: enabled Jul 11 14:39:22.058933 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Jul 11 14:39:22.058933 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 11 14:39:22.111834 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: Python main interpreter initialized at 0x55fdb85179c0 Jul 11 14:39:22.111834 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: python threads support enabled Jul 11 14:39:22.111834 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: your server socket listen backlog is limited to 100 connections Jul 11 14:39:22.111834 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: your mercy for graceful operations on workers is 90 seconds Jul 11 14:39:22.112285 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: mapped 403077 bytes (393 KB) for 2 cores Jul 11 14:39:22.112380 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: *** Operational MODE: preforking *** Jul 11 14:39:22.112615 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: *** uWSGI is running in multiple interpreter mode *** Jul 11 14:39:22.112615 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: spawned uWSGI master process (pid: 130142) Jul 11 14:39:22.112831 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Jul 11 14:39:22.113439 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: spawned uWSGI worker 1 (pid: 130159, cores: 1) Jul 11 14:39:22.113793 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: spawned uWSGI worker 2 (pid: 130160, cores: 1) Jul 11 14:39:22.113947 dv-cinder-master-3-iscsi devstack@n-api.service[130142]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 11 14:39:22.792673 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 11 14:39:22.795028 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Jul 11 14:39:24.008587 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 14:39:24.008587 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: from cryptography.utils import int_from_bytes Jul 11 14:39:24.009813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 14:39:24.009813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: from cryptography.utils import int_from_bytes Jul 11 14:39:24.069046 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 14:39:24.069046 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: from cryptography.utils import int_from_bytes Jul 11 14:39:24.069714 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 14:39:24.069714 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: from cryptography.utils import int_from_bytes Jul 11 14:39:26.394152 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 11 14:39:26.395194 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 11 14:39:26.395194 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 11 14:39:26.395362 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 11 14:39:26.395564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 11 14:39:26.396323 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.396888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.397389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.397935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.398492 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.399003 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.399584 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.400082 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.400674 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.401115 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.401599 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.402169 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.402706 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.403216 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.403701 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.404220 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.404690 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.405193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.405675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.406499 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.407006 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.407475 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.407946 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.408499 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.409106 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.409664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.410200 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.410748 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.411289 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.411799 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.412428 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.412964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.413365 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.413775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.414219 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.414771 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.415083 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.415426 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.415761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.416084 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.416451 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.416768 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.417103 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.417435 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.417752 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.418074 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.418394 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.418924 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.419422 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.419782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.420049 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.420367 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.420884 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.421222 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.421565 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.421827 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.422088 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.422983 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.423422 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.423773 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.424166 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.424562 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.424915 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.425244 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.425569 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.425941 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.426315 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.426753 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.427287 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.427702 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.428034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.428371 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.428843 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.72 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.429154 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.72 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.429520 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.429935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.430269 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.430681 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.431020 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.431354 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.431675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.432228 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.432605 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.432926 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.433269 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.433800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.434114 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.434445 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.434970 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.435991 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.436312 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.436631 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.436959 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.437258 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.437570 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.437883 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.438255 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.438629 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.439047 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.439525 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.440014 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.440329 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.440656 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.440972 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.441300 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.441614 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.441929 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.442241 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.442573 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.442899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.443211 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.443536 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.443850 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.444159 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.444474 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.444796 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.445110 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.445418 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.445737 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.446044 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.446352 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.446711 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.447051 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.447367 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.448054 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.448307 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.448549 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.449000 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.449182 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.449356 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.449617 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.450109 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.450465 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.450839 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.451725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.452037 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.452363 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.452708 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.453052 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.453399 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.453718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.454057 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.454391 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.455056 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.455390 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.455707 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.456036 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.456426 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.456662 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.456978 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.457478 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.457847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.458144 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.458470 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.458796 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.459118 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.459583 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.460164 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.460475 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.461019 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.461297 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.461657 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.462006 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.462333 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.462695 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.463084 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.463409 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.463725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.464137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.464365 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.464705 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.465058 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.465348 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.465664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.466021 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.466328 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.466706 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.466963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.467308 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.467630 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.468402 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.468402 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.468883 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.468883 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.469176 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.469481 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.469788 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.471123 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.471123 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.471123 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.471123 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.471876 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.471876 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.472317 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.472317 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.472756 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.472928 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.473165 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.473483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.473859 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.474102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.474414 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.474599 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 11 14:39:26.475158 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.476041 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 11 14:39:26.476365 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.476531 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 11 14:39:26.476738 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.476738 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.477185 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 11 14:39:26.477341 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.477494 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 11 14:39:26.477651 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.477811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.477972 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.478118 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.478261 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.478491 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.478852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.479156 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.479315 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.479462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.480221 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.480814 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.480985 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.481184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.481343 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.481499 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.481651 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.481803 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.481948 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.482156 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.482422 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.482607 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.482753 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.482907 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.483068 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.483233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.483379 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.483530 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.483676 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.483819 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.483979 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.484123 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.484266 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.484659 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.484811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.484811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.485102 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.485259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.485426 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.485580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.485580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.485861 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.486035 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.486181 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.486353 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.486657 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.486824 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.486980 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.487280 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.487444 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.487630 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.488012 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.488175 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.488351 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.488502 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.488655 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.488825 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.489079 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.489408 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.489569 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.489714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.489892 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.490057 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.490057 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.490481 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.491074 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.491226 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.491370 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.491513 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.491669 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.491821 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.491989 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.492135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.492287 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.492464 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.492732 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.492892 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.493038 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.493038 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.493318 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.493468 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.493621 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.493621 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.493902 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.494057 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.494208 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.494365 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.494554 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.494813 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.494972 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.495166 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.495324 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.495324 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.495614 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.495758 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.495901 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.496053 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.496211 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.496355 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.496507 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.496813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.496975 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.497127 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.497442 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.497594 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.498293 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.498467 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.498619 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.498764 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.498927 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.499189 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.499362 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.499528 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.499673 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.499834 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.499978 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.499978 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.500275 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.500428 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.500572 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.500572 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.501222 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.501441 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.501588 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.501751 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.501904 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.502048 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.502200 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.502352 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.502548 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.502548 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.502964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.503108 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.503339 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.503494 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.503681 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.503952 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.504118 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.504272 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.504433 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.504580 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.504580 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.504961 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.505111 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.505264 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.505419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.505572 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.505572 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.505855 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.72 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.506011 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.506155 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.72 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.506311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.506487 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.506645 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.506799 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.506952 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.507227 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.507373 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.507655 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.507809 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.507962 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.508107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.508276 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.508421 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.508573 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.508726 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.508879 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.509136 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.509413 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.509575 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.509725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.509878 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.510080 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.510226 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.510381 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.510645 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.510801 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.511380 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.511669 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.511835 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.511989 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.512142 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.512287 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.512430 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.512581 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.512736 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.513020 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.513172 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.513424 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.513584 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.513730 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.513885 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.514031 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.514175 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.514318 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.514570 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.514724 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.514888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.515157 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.515157 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.515449 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.515611 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.515767 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.515922 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.516074 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.516227 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.516384 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.516529 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.516686 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.516838 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.516983 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.517294 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.517471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.517662 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.517955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.518162 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.518316 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.518554 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.518708 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.518708 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.519047 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.519214 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.519490 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.519697 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.519853 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.520013 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.520182 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.520328 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.520480 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.520682 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.520829 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.520996 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.520996 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.521785 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.521947 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.522135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.522280 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.522441 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.522619 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.522780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.522780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.522780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.523381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.523528 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.523680 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.523834 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.523993 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.524146 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.524298 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.524442 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.524442 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.524716 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.524885 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.525030 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.525290 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.525454 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.525608 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.525608 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.525893 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.526047 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.526193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.526338 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.526524 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.527342 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.527671 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.527834 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.528155 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.528316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.528316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.528612 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.528758 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.528941 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.529102 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 14:39:26.529256 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.529541 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.529694 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.529840 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.530005 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.530165 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.530317 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.530480 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.530634 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.530791 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.530791 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.531082 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.531082 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.531766 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.531925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.532070 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.532225 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.532380 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.532532 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.532691 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.532837 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.532989 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.533255 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.533414 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.533559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.533711 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.533863 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.534016 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.534171 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.534335 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.534614 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.534768 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.534913 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.535058 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.535407 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.535568 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.535715 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.535993 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.536153 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.536299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.536453 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.536598 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.536753 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.536898 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.537041 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.537200 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.537345 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.537497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.537792 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.537960 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.538107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.538403 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.538595 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.538742 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.538907 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.538907 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.539197 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.539507 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.539768 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.539915 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.540067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.540221 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.540376 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.540520 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.540672 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.540891 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.541059 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.541241 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.541399 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.541976 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.542147 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.542313 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.542546 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.542706 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.542872 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.543025 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.543187 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.543465 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.543624 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.543784 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.543965 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.544126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.544285 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.544437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.545119 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.545284 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.545467 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.545732 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.545889 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.546049 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.546207 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.546367 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.546558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.546724 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.546876 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.547026 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.547175 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.547335 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.547493 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.547652 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.547819 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.548108 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.548265 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.548570 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.548729 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.548890 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.549060 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.549230 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.549873 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.550006 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.550096 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.550180 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.550265 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.550346 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.550448 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.550527 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.550599 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.550669 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.550754 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.550828 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.550907 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.550986 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.551066 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.551139 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.551216 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.551216 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.551354 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.551426 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.551496 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.551576 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.551654 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.551731 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.551816 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.551918 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.552087 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.552196 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.552308 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.552439 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.552540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.552625 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.552698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.552771 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.552851 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.552935 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.553017 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.553139 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.553260 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.553339 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.553339 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.553493 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.553569 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.553655 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.553729 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.553800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.553800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.553938 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.554012 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.554084 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.554155 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.554233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.554315 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.554395 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.554502 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.554578 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.554649 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.554735 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.554822 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.554896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.554974 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.555046 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.555116 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.555193 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.555293 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.555381 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.555381 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.555529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.555638 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.555730 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.555925 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.556057 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.556159 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.556247 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.556341 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.556442 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.556530 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.556616 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.556701 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.556792 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.556886 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.557022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.557112 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.557197 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.557283 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.557375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.557462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.557560 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.557655 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.557742 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.557834 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.557920 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.558018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.558105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.558237 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.558331 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.558417 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.558538 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.558623 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.558623 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.558623 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.558886 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.558989 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.559076 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.559176 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.559289 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.559385 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.559481 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.559569 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.559651 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.559746 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.559856 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.559951 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.560038 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.560123 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.560256 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.560354 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.560447 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.560566 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.560669 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.560757 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.560855 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.560950 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.561037 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.561129 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.561129 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.561292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.561457 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.561554 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.561641 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.561726 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.561811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.561912 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.561997 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.562070 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.562149 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.562222 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.562293 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.562365 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.562463 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.562552 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.562634 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.562707 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.562786 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.562882 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.562977 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.563070 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.563157 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.563241 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.563340 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.563441 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.563528 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.563770 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.563885 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.564081 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.564176 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.564393 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.564502 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.564710 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.564806 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.565025 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.565119 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.565344 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.565439 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.565659 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.565751 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.565965 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.566078 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.566284 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.566617 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.566754 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.566858 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.566947 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.567109 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.567281 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.567377 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.567599 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.567694 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.567914 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.568063 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.568229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.568329 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.568545 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.568634 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.568870 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.569009 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.569210 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.569317 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.569710 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.569815 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.569909 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.569986 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.574538 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.574538 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.574859 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.574988 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.575117 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.575242 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.575364 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.575484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.575484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.575484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.576376 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.576600 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.576724 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.576815 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.576911 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.576997 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.577094 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.577189 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.577189 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.577412 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.577503 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.577614 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.577763 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.577860 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.577959 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.578055 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.578193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.578299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.578459 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.578594 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.578698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.578698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.578698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.578917 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.578999 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.578999 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.579130 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.579222 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.579309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.579403 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.579814 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.579932 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.579932 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.580075 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.580172 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.580267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.580489 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.580610 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.580898 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.580995 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.581330 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.581749 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.581853 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.582161 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.582519 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.582770 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.582949 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.583123 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.583295 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.583488 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.583776 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.583938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.584165 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.584304 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.584445 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.584538 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.584895 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.585078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.585354 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.585520 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.585731 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.585838 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.586170 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.586366 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.586832 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.587062 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.587221 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.587877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.588001 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.588106 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.588205 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.588344 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.588439 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.588650 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.588787 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.588964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.589109 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.589282 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.589417 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.589596 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.589881 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.590034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.590210 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.590306 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.590558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.590657 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.590879 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.590976 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.591202 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.591303 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.591586 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.592300 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.592485 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.592822 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.592924 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.593145 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.593272 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.593524 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.593654 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.593853 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.594014 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.594181 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.594347 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.594521 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.594703 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.594836 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.595047 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.595158 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.595387 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.595491 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.595747 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.595850 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.596061 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.596163 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.596368 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.596470 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.596680 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.596809 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.596971 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.597076 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.597291 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.597397 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.597598 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.597701 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.597913 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.598071 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.598223 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.598327 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.598553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.598664 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.598873 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.599161 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.599257 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.599475 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.599570 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.599825 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.599936 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.600216 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.600323 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.600476 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.600615 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.600805 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.600909 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.601132 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.601226 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.601466 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.601568 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.601783 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.601881 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.602110 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.602206 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.602423 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.602573 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.602860 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.603169 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.603169 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.603450 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.603550 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.603658 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.603772 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.604061 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.604169 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.604312 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.604447 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.604595 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.604735 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.604831 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.604918 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.605129 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.605251 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.605471 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.605614 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.605779 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.606059 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.606185 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.606385 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.606588 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.606727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.606830 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.607071 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.607168 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.607391 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.607498 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.607787 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.607884 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.608104 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.608200 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.608521 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.608871 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.609192 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.609369 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.609556 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.609711 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.609876 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.610037 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.610192 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.610393 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.610647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.610817 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.611025 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.611183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.611346 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.611639 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.611824 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.611990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.612145 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.612306 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.612472 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.612666 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.612823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.612986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.613140 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.613304 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.613457 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.613610 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.613789 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.613958 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.614120 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.614280 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.614594 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.614718 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.614820 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.614907 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.615000 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.615087 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.615233 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.615343 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.615552 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.615663 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.615866 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.615969 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.616178 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.616274 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.616490 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.616585 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.616805 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.616928 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.617369 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.617520 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.617616 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.617766 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.617994 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.618148 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.618327 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.618513 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.618686 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.619021 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.619334 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.619793 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.619946 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.620071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.620204 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.620447 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.620644 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.620905 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.621180 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.621284 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.621505 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.621599 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.621837 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.622042 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.622168 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.622292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.622512 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.622666 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.622830 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.623011 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.623206 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.623313 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.623464 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.623611 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.623787 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.623891 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.624098 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.624227 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.624410 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.624559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.624722 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.624819 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.625031 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.625133 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.625356 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.625459 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.625672 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.625769 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.625981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.626083 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.626301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.626397 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.626638 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.626744 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.627014 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.627183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.627317 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.627636 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.627875 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.627980 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.628199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.628294 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.628556 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.628664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.628924 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.629046 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.629233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.629331 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.629561 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.629879 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.629981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.630223 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.630370 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.630578 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.630684 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.630905 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.631002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.631224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.631327 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.631530 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.631633 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.632340 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.632458 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.632567 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.632664 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.632751 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.632852 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.632956 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.633088 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.633209 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.633305 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.633380 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.633479 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.633629 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.633725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.633951 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.634052 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.634265 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.634413 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.634737 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.634832 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.634910 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.635059 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.635242 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.635403 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.635564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.635711 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.635856 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.636006 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.636158 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.636323 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.636474 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.636618 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.636770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.636912 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.637067 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.637224 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.637366 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.637517 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.637659 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.637815 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.637923 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.638069 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.638171 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.638969 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.639065 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.639335 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.639445 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.639665 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.639769 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.640044 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.640203 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.642318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.642522 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.642648 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.642772 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.642897 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.642897 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.642897 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.642897 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.643331 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.643444 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.643557 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.643676 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.643803 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.643803 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.644016 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.644127 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.644237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.644372 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.644614 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.644735 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.644866 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.645079 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.645207 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.645335 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.645448 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.645566 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.645678 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.645813 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.645938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.646065 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.646186 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.646317 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.646447 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.646565 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.646677 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.650686 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.650790 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.650894 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.651044 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.651155 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.651242 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.651242 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.651397 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.651480 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.651554 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.651626 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.651707 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.651781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.651861 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.651935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.652008 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.652087 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.652171 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.652245 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.652324 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.652399 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.652472 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.652550 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.652633 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.652708 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.652788 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.652862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.652936 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.653021 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.653103 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.653176 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.653247 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.653318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.653398 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.653483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.653589 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.653665 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.653736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.653819 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.653949 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.654048 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.654660 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.654749 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.654825 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.654897 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.654979 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.654979 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.655594 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.655594 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.656034 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.656252 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.657067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.657067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.657229 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.657415 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.657776 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.658042 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.658410 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.658704 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.659022 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.659345 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.659657 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.659963 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.660272 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.660556 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.660699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.660998 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.661299 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.661610 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.661869 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.662213 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.662897 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.662991 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.663077 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.663077 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.663233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.663309 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.663389 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.663462 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.663536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.664387 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.664588 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.664760 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.664968 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.665158 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.665333 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.665626 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.665787 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.665941 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.666103 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.666103 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.666454 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.666630 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.666784 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.667353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.667527 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.667715 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.667934 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.667934 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.668349 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.668523 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.668687 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.668853 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.669019 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.669184 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.669405 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] pci.passthrough_whitelist = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.669405 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.669405 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.669405 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.669405 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.669405 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.671042 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.671042 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.671042 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.671042 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.671042 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.671932 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.671932 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.671932 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.672388 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.672388 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.672843 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.673745 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.673745 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.674130 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.674313 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.674559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.674742 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.674906 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.675060 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.675223 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.675391 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] powervm.disk_driver = localdisk {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.675556 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.675884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] powervm.proc_units_factor = 0.1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.676054 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.676218 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] powervm.volume_group_name = {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.676386 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.676543 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.676713 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.676875 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.677060 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.677650 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.677824 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.678040 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.678218 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.678509 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.678675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.678838 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.679007 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.679161 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.679313 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.679539 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.679710 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.679996 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.680170 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.680326 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.680480 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.680633 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.680794 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.680961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.681116 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.681269 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.681430 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.681640 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.681827 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.681827 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.681827 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.682339 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.682529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.682698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.682862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.682862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.683172 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.683172 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.683480 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.683480 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.683780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.683780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.684126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.684325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.684481 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.684642 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.684808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.684973 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.685155 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.685323 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.685486 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.685641 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.685808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.686084 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.686245 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.686245 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.686589 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.686756 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.686921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.687085 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.687239 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.687677 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.687942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.688171 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.688335 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.688509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.688745 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.688915 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.689077 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.689242 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.689397 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.689592 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.689776 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.689938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.690200 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.690369 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.690573 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.690729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.690908 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.691076 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.691241 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.691395 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.691557 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.691764 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.691926 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.692138 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.692309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.692660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.692841 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.692999 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.693153 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.693329 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.693492 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.693646 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.693800 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.693962 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.694124 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.694325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.694532 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.694691 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.694856 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.695012 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.695182 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.695352 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.695521 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.695675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.695838 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.695993 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.696374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.696623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.696623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.696929 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.697085 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.697239 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.697403 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.697871 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.698218 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.698500 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.698667 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=130159) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 11 14:39:26.698953 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.698953 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.698953 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.698953 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.698953 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.698953 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.698953 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.698953 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.700217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.700217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.700217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.700217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.700217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.700217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.700217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.700217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.702150 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.702150 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.702150 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.702694 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.702694 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.702998 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.703215 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.703529 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.703854 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.704179 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.704860 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.705047 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.705369 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.705682 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.705994 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.706311 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.706656 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.706976 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.707289 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.707602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.708125 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.708288 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.708495 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.708926 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.709285 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.709526 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.709814 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.710152 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.710572 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.710810 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.711136 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.711447 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.711763 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.712136 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.712486 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.712908 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.713272 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.713604 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.713924 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.714259 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.714616 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.715162 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.715351 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.716253 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.716253 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.716694 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.717001 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.717248 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.717585 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.718210 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.718394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.718655 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.719015 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.719335 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.719612 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.719922 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.720245 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.720561 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.720874 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.721194 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.721504 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.721846 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.722253 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.722599 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.722787 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.723068 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.723388 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.723749 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.724080 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.724400 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.724977 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.725826 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.725826 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.725826 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.726552 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.726552 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.726856 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.727044 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.727350 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.727666 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.728139 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.728577 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.728775 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.729054 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.729398 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.729687 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.730028 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.730343 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.730702 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.731020 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.731361 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.731703 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.732579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.732915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.733229 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.733541 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.733851 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.734171 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.734532 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.734851 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.735163 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.735475 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.735787 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.736101 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.736483 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.736705 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.737013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.737485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.737667 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.737932 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.738272 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.738778 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.738957 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.739164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.739501 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.739796 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.740095 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.740407 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.740716 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.741393 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.741393 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.741740 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.742173 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.743020 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.743020 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.743020 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.743494 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.743647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.743884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.744210 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.744532 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.744867 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.745183 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.745507 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.745816 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.746130 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.746610 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.746799 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.747104 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.747432 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.747835 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.748135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.748461 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.748897 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.749102 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.749418 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.749757 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.750070 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.750394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.750738 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.751053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.751369 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.751691 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.752011 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.752348 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.752716 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.752987 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.753317 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.753645 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.754004 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.754323 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.754707 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.755035 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.755353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.755777 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.755982 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 14:39:26.756221 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=130160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 11 14:39:26.852745 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO dbcounter [None req-33968a65-c13b-4ba1-886f-486255433858 None None] Registered counter for database nova_api Jul 11 14:39:26.875615 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_db.sqlalchemy.engines [None req-33968a65-c13b-4ba1-886f-486255433858 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130159) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 14:39:26.883685 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writer thread running {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 14:39:26.915597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO dbcounter [None req-e854ee50-bda7-4e2d-9bbb-9f898b587c22 None None] Registered counter for database nova_api Jul 11 14:39:26.936604 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_db.sqlalchemy.engines [None req-e854ee50-bda7-4e2d-9bbb-9f898b587c22 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130160) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 14:39:26.943730 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writer thread running {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 14:39:26.944701 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.context [None req-33968a65-c13b-4ba1-886f-486255433858 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),14f12807-d728-487f-b4ea-df7168086c51(cell1) {{(pid=130159) load_cells /opt/stack/nova/nova/context.py:464}} Jul 11 14:39:26.949614 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-33968a65-c13b-4ba1-886f-486255433858 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:39:26.951287 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-33968a65-c13b-4ba1-886f-486255433858 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:39:26.952101 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-33968a65-c13b-4ba1-886f-486255433858 None None] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:39:26.954406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-33968a65-c13b-4ba1-886f-486255433858 None None] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:39:26.956760 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO dbcounter [None req-33968a65-c13b-4ba1-886f-486255433858 None None] Registered counter for database nova_cell0 Jul 11 14:39:26.960921 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO dbcounter [None req-33968a65-c13b-4ba1-886f-486255433858 None None] Registered counter for database nova_cell1 Jul 11 14:39:26.969828 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_db.sqlalchemy.engines [None req-33968a65-c13b-4ba1-886f-486255433858 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130159) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 14:39:26.971292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_db.sqlalchemy.engines [None req-33968a65-c13b-4ba1-886f-486255433858 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130159) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 14:39:26.982847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writer thread running {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 14:39:26.985723 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writer thread running {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 14:39:27.003888 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.context [None req-e854ee50-bda7-4e2d-9bbb-9f898b587c22 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),14f12807-d728-487f-b4ea-df7168086c51(cell1) {{(pid=130160) load_cells /opt/stack/nova/nova/context.py:464}} Jul 11 14:39:27.007984 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e854ee50-bda7-4e2d-9bbb-9f898b587c22 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:39:27.009632 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e854ee50-bda7-4e2d-9bbb-9f898b587c22 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:39:27.010500 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e854ee50-bda7-4e2d-9bbb-9f898b587c22 None None] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:39:27.012518 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e854ee50-bda7-4e2d-9bbb-9f898b587c22 None None] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:39:27.015277 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO dbcounter [None req-e854ee50-bda7-4e2d-9bbb-9f898b587c22 None None] Registered counter for database nova_cell0 Jul 11 14:39:27.021745 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO dbcounter [None req-e854ee50-bda7-4e2d-9bbb-9f898b587c22 None None] Registered counter for database nova_cell1 Jul 11 14:39:27.031290 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_db.sqlalchemy.engines [None req-e854ee50-bda7-4e2d-9bbb-9f898b587c22 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130160) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 14:39:27.032088 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_db.sqlalchemy.engines [None req-e854ee50-bda7-4e2d-9bbb-9f898b587c22 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130160) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 14:39:27.039418 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writer thread running {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 14:39:27.040947 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writer thread running {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 14:39:27.192192 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO dbcounter [None req-e854ee50-bda7-4e2d-9bbb-9f898b587c22 None None] Registered counter for database nova_cell0 Jul 11 14:39:27.199804 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_db.sqlalchemy.engines [None req-e854ee50-bda7-4e2d-9bbb-9f898b587c22 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130160) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 14:39:27.204737 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writer thread running {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 14:39:27.235287 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO dbcounter [None req-33968a65-c13b-4ba1-886f-486255433858 None None] Registered counter for database nova_cell0 Jul 11 14:39:27.242876 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_db.sqlalchemy.engines [None req-33968a65-c13b-4ba1-886f-486255433858 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=130159) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 14:39:27.250265 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writer thread running {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 14:39:27.535939 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: WARNING keystonemiddleware.auth_token [None req-e854ee50-bda7-4e2d-9bbb-9f898b587c22 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 11 14:39:27.543468 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: WARNING keystonemiddleware.auth_token [None req-e854ee50-bda7-4e2d-9bbb-9f898b587c22 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 11 14:39:27.581923 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: WARNING keystonemiddleware.auth_token [None req-33968a65-c13b-4ba1-886f-486255433858 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 11 14:39:27.589767 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: WARNING keystonemiddleware.auth_token [None req-33968a65-c13b-4ba1-886f-486255433858 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 11 14:39:27.892446 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: WARNING keystonemiddleware.auth_token [None req-e854ee50-bda7-4e2d-9bbb-9f898b587c22 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 11 14:39:27.898793 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: WARNING keystonemiddleware.auth_token [None req-e854ee50-bda7-4e2d-9bbb-9f898b587c22 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 11 14:39:27.904916 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55fdb85179c0 pid: 130160 (default app) Jul 11 14:39:27.941491 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: WARNING keystonemiddleware.auth_token [None req-33968a65-c13b-4ba1-886f-486255433858 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Jul 11 14:39:27.947777 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: WARNING keystonemiddleware.auth_token [None req-33968a65-c13b-4ba1-886f-486255433858 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Jul 11 14:39:27.953309 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55fdb85179c0 pid: 130159 (default app) Jul 11 14:40:09.618400 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=3 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:40:09.619264 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=3 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:40:09.620214 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=3 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:40:09.621451 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=3 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:40:11.231605 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3146637a-8ac4-4587-9df7-9f4125450b04 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Action: 'create', calling method: .version_select at 0x7f59318c55e0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1889578011"}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:40:13.005383 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-3146637a-8ac4-4587-9df7-9f4125450b04 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 3.401572 Jul 11 14:40:13.006021 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 1/1] 10.222.0.72 () {60 vars in 1154 bytes} [Mon Jul 11 14:40:09 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 3404 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) Jul 11 14:40:13.023383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=3 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:40:13.024526 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=3 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:40:13.027040 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=3 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:40:13.029016 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=3 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:40:13.434366 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-5f70e344-959e-4ad0-b893-98f8c5623a54 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Calling method '.version_select at 0x7f59319581f0>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:13.969462 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-5f70e344-959e-4ad0-b893-98f8c5623a54 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "GET /compute/v2.1/os-volumes/93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f" status: 200 len: 346 microversion: 2.1 time: 0.953716 Jul 11 14:40:13.977231 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 1/2] 10.222.0.72 () {58 vars in 1244 bytes} [Mon Jul 11 14:40:13 2022] GET /compute/v2.1/os-volumes/93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f => generated 346 bytes in 957 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:40:13.986355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-fdaf75ed-aa13-4dd3-85cd-e5c96d9c3ad4 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Action: 'create', calling method: .version_select at 0x7f593178b430>, body: {"snapshot": {"volume_id": "93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1204304459"}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:40:14.544688 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-fdaf75ed-aa13-4dd3-85cd-e5c96d9c3ad4 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.565845 Jul 11 14:40:14.550063 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 2/3] 10.222.0.72 () {60 vars in 1161 bytes} [Mon Jul 11 14:40:13 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:40:14.555805 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3efe2efe-048b-4aae-91c6-3ab527d1a91e tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Calling method '.version_select at 0x7f593181f4c0>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:14.609761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3efe2efe-048b-4aae-91c6-3ab527d1a91e tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2" status: 200 len: 287 microversion: 2.1 time: 0.058298 Jul 11 14:40:14.610389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 2/4] 10.222.0.72 () {58 vars in 1250 bytes} [Mon Jul 11 14:40:14 2022] GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 => generated 287 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:40:14.620437 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-da329887-c41d-44ad-83de-51f77d508722 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Calling method '.version_select at 0x7f593178b4c0>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:14.683158 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-da329887-c41d-44ad-83de-51f77d508722 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.066499 Jul 11 14:40:14.684955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 3/5] 10.222.0.72 () {58 vars in 1139 bytes} [Mon Jul 11 14:40:14 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:40:14.696670 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1d290aca-0408-427b-a287-7bac3cbfb4d0 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Calling method '.version_select at 0x7f593181fc10>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:14.731571 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1d290aca-0408-427b-a287-7bac3cbfb4d0 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2" status: 200 len: 287 microversion: 2.1 time: 0.039483 Jul 11 14:40:14.732695 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 3/6] 10.222.0.72 () {58 vars in 1250 bytes} [Mon Jul 11 14:40:14 2022] GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 => generated 287 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:40:15.746419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2bdcb46f-8f8a-41d3-8498-6f3c671d22fb tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Calling method '.version_select at 0x7f59317a51f0>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:15.809520 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2bdcb46f-8f8a-41d3-8498-6f3c671d22fb tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2" status: 200 len: 287 microversion: 2.1 time: 0.068018 Jul 11 14:40:15.810590 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 4/7] 10.222.0.72 () {58 vars in 1250 bytes} [Mon Jul 11 14:40:15 2022] GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 => generated 287 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:40:16.821864 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8a087c3a-726e-486f-9299-9012421a3108 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Calling method '.version_select at 0x7f593181fdc0>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:16.878998 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8a087c3a-726e-486f-9299-9012421a3108 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2" status: 200 len: 287 microversion: 2.1 time: 0.060900 Jul 11 14:40:16.880125 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 4/8] 10.222.0.72 () {58 vars in 1250 bytes} [Mon Jul 11 14:40:16 2022] GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 => generated 287 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:40:17.891597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9c8dde7a-935f-44e0-a319-74acfa19fcf5 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Calling method '.version_select at 0x7f59317a5940>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:18.007504 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9c8dde7a-935f-44e0-a319-74acfa19fcf5 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2" status: 200 len: 287 microversion: 2.1 time: 0.120428 Jul 11 14:40:18.012204 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 5/9] 10.222.0.72 () {58 vars in 1250 bytes} [Mon Jul 11 14:40:17 2022] GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 => generated 287 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:40:19.020081 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-18657602-ae4f-4524-8d06-9bb3eed7dc6c tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Calling method '.version_select at 0x7f593181f550>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:19.125224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-18657602-ae4f-4524-8d06-9bb3eed7dc6c tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2" status: 200 len: 287 microversion: 2.1 time: 0.109629 Jul 11 14:40:19.126313 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 5/10] 10.222.0.72 () {58 vars in 1250 bytes} [Mon Jul 11 14:40:19 2022] GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 => generated 287 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:40:20.137557 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-89b5a79e-3753-4a5a-8538-6fdd37565c19 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Calling method '.version_select at 0x7f593178bd30>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:20.242526 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-89b5a79e-3753-4a5a-8538-6fdd37565c19 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2" status: 200 len: 287 microversion: 2.1 time: 0.108617 Jul 11 14:40:20.243616 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 6/11] 10.222.0.72 () {58 vars in 1250 bytes} [Mon Jul 11 14:40:20 2022] GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 => generated 287 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:40:21.255640 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-59632c26-b207-484d-82d9-93667ae9b731 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Calling method '.version_select at 0x7f59317df3a0>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:21.339191 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-59632c26-b207-484d-82d9-93667ae9b731 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2" status: 200 len: 288 microversion: 2.1 time: 0.087236 Jul 11 14:40:21.340857 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 6/12] 10.222.0.72 () {58 vars in 1250 bytes} [Mon Jul 11 14:40:21 2022] GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 => generated 288 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:40:21.357568 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-0fe9fc4f-8275-42dc-89be-4b36f4ed281d tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Calling method '.version_select at 0x7f593178b8b0>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:21.518810 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-0fe9fc4f-8275-42dc-89be-4b36f4ed281d tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "DELETE /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2" status: 202 len: 0 microversion: 2.1 time: 0.164714 Jul 11 14:40:21.520565 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 7/13] 10.222.0.72 () {58 vars in 1253 bytes} [Mon Jul 11 14:40:21 2022] DELETE /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 => generated 0 bytes in 169 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:40:21.529118 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-4b74cfd3-6822-4b1a-a857-637e2092f608 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Calling method '.version_select at 0x7f59317dfa60>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:21.717111 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-4b74cfd3-6822-4b1a-a857-637e2092f608 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2" status: 200 len: 287 microversion: 2.1 time: 0.191960 Jul 11 14:40:21.717920 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 7/14] 10.222.0.72 () {58 vars in 1250 bytes} [Mon Jul 11 14:40:21 2022] GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 => generated 287 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:40:22.430788 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f37911d3-f5d9-4cbf-8eaf-a10ef2ec8390 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1721696206", "imageRef": "a588fa24-23b5-4e53-87c6-7e600ee3a5ef", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:40:22.646895 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.network.neutron [None req-f37911d3-f5d9-4cbf-8eaf-a10ef2ec8390 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] validate_networks() for [('382f35eb-d1cd-45e3-b28b-3e3389f544db', None, None, None, None, None)] {{(pid=130160) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:40:22.733126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a463e68f-b4b6-429c-9b5e-86096d673e0f tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Calling method '.version_select at 0x7f593181f4c0>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:22.803845 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-a463e68f-b4b6-429c-9b5e-86096d673e0f tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] HTTP exception thrown: Snapshot 8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 could not be found. Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a463e68f-b4b6-429c-9b5e-86096d673e0f tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Request method failure captured: Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 HTTP/1.1 Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZJgVUDWF113vWklHc3RJdVt8DJ5YCrk62yu2we2zhoIr7aroHG3AXLb2h8RF3djojz7w9cKsSG-Xko_jKZiXDi8fOLP4LWkealDsRoZhDOnxD1Lqrct4MlMnTIF0glAgCfNvtnoMOSrWPLZlmsT0lE1cm3nBpD509RW7WY44Wu4Rg Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: bf9d11f0b33a402f8584b4a158b3b41b Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-142704109 Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:40:22.811292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:40:22.812430 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/bf9d11f0b33a402f8584b4a158b3b41b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/bf9d11f0b33a402f8584b4a158b3b41b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_bf9d11f0b33a402f8584b4a158b3b41b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/bf9d11f0b33a402f8584b4a158b3b41b"}]}] Jul 11 14:40:22.812430 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-VolumesSnapshotsTestJSON-142704109 Jul 11 14:40:22.812430 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: bf9d11f0b33a402f8584b4a158b3b41b Jul 11 14:40:22.812430 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-142704109 Jul 11 14:40:22.812430 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-VolumesSnapshotsTestJSON-142704109-project Jul 11 14:40:22.812430 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:40:22.812430 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:40:22.812430 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 072255e7ae5147cb9a1540fd12f242da Jul 11 14:40:22.812430 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-VolumesSnapshotsTestJSON-142704109-project Jul 11 14:40:22.812430 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: Jul 11 14:40:22.812430 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Snapshot 8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 could not be found. Jul 11 14:40:22.812430 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2'} Jul 11 14:40:22.812430 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:40:22.813126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:40:22.813126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Jul 11 14:40:22.813126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Jul 11 14:40:22.813126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Jul 11 14:40:22.813126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Jul 11 14:40:22.813126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Jul 11 14:40:22.813126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Jul 11 14:40:22.813126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 11 14:40:22.813126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 11 14:40:22.813126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 11 14:40:22.813126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 11 14:40:22.813126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 11 14:40:22.813126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 11 14:40:22.813126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 11 14:40:22.813126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 11 14:40:22.813126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 could not be found. (HTTP 404) (Request-ID: req-445c3dd4-bcd4-4105-845d-94699705e69b) Jul 11 14:40:22.813126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:22.813126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 11 14:40:22.814022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 could not be found. Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Jul 11 14:40:22.814938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:40:22.815889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 could not be found. Jul 11 14:40:22.815889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:22.815889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:40:22.815889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:22.815889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:40:22.815889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:40:22.815889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:40:22.815889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:40:22.815889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:40:22.815889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 could not be found. Jul 11 14:40:22.815889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:22.815889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a463e68f-b4b6-429c-9b5e-86096d673e0f tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Returning 404 to user: Snapshot 8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:40:22.815889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a463e68f-b4b6-429c-9b5e-86096d673e0f tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2" status: 404 len: 111 microversion: 2.1 time: 0.087417 Jul 11 14:40:22.817332 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 8/15] 10.222.0.72 () {58 vars in 1250 bytes} [Mon Jul 11 14:40:22 2022] GET /compute/v2.1/os-snapshots/8e95e3eb-64e1-4d68-bc7b-858f7b6b40a2 => generated 111 bytes in 91 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:40:22.826393 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-64ba6f64-7d51-47e4-befd-7e7d7095caa1 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Calling method '.version_select at 0x7f59321b64c0>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:22.943198 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-64ba6f64-7d51-47e4-befd-7e7d7095caa1 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "DELETE /compute/v2.1/os-volumes/93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f" status: 202 len: 0 microversion: 2.1 time: 0.122904 Jul 11 14:40:22.944325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 9/16] 10.222.0.72 () {58 vars in 1247 bytes} [Mon Jul 11 14:40:22 2022] DELETE /compute/v2.1/os-volumes/93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f => generated 0 bytes in 125 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:40:22.954149 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-6488ba78-13da-4dfa-ad3a-bb5feb788fc8 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Calling method '.version_select at 0x7f593181f3a0>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:23.098402 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-6488ba78-13da-4dfa-ad3a-bb5feb788fc8 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "GET /compute/v2.1/os-volumes/93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f" status: 200 len: 345 microversion: 2.1 time: 0.147560 Jul 11 14:40:23.100077 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 10/17] 10.222.0.72 () {58 vars in 1244 bytes} [Mon Jul 11 14:40:22 2022] GET /compute/v2.1/os-volumes/93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f => generated 345 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:40:24.113586 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-5ee878e0-3e42-48ff-94a3-a78bc053916d tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Calling method '.version_select at 0x7f59317ba5e0>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:24.197483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-5ee878e0-3e42-48ff-94a3-a78bc053916d tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] HTTP exception thrown: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-5ee878e0-3e42-48ff-94a3-a78bc053916d tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Request method failure captured: Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/os-volumes/93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f HTTP/1.1 Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZJgVUDWF113vWklHc3RJdVt8DJ5YCrk62yu2we2zhoIr7aroHG3AXLb2h8RF3djojz7w9cKsSG-Xko_jKZiXDi8fOLP4LWkealDsRoZhDOnxD1Lqrct4MlMnTIF0glAgCfNvtnoMOSrWPLZlmsT0lE1cm3nBpD509RW7WY44Wu4Rg Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: bf9d11f0b33a402f8584b4a158b3b41b Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-142704109 Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:40:24.200865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:40:24.202363 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/bf9d11f0b33a402f8584b4a158b3b41b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/bf9d11f0b33a402f8584b4a158b3b41b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_bf9d11f0b33a402f8584b4a158b3b41b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/bf9d11f0b33a402f8584b4a158b3b41b"}]}] Jul 11 14:40:24.202363 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-VolumesSnapshotsTestJSON-142704109 Jul 11 14:40:24.202363 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: bf9d11f0b33a402f8584b4a158b3b41b Jul 11 14:40:24.202363 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-142704109 Jul 11 14:40:24.202363 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-VolumesSnapshotsTestJSON-142704109-project Jul 11 14:40:24.202363 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:40:24.202363 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:40:24.202363 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 072255e7ae5147cb9a1540fd12f242da Jul 11 14:40:24.202363 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-VolumesSnapshotsTestJSON-142704109-project Jul 11 14:40:24.202363 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: Jul 11 14:40:24.202363 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. Jul 11 14:40:24.202363 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f'} Jul 11 14:40:24.202363 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. (HTTP 404) (Request-ID: req-dc13e9e9-2f60-4cbd-aa8b-5ee99ca0f29f) Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:40:24.203224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 11 14:40:24.204107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 11 14:40:24.205318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:40:24.206214 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. Jul 11 14:40:24.206214 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.206214 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:40:24.206214 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.206214 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:40:24.206214 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:40:24.206214 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:40:24.206214 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:40:24.206214 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:40:24.206214 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. Jul 11 14:40:24.206214 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.209649 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-5ee878e0-3e42-48ff-94a3-a78bc053916d tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Returning 404 to user: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:40:24.211225 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-5ee878e0-3e42-48ff-94a3-a78bc053916d tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "GET /compute/v2.1/os-volumes/93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f" status: 404 len: 109 microversion: 2.1 time: 0.102331 Jul 11 14:40:24.212225 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 11/18] 10.222.0.72 () {58 vars in 1244 bytes} [Mon Jul 11 14:40:24 2022] GET /compute/v2.1/os-volumes/93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f => generated 109 bytes in 104 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 11 14:40:24.231876 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f8a0fac3-d750-4df9-81b9-06b38fe49799 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Calling method '.version_select at 0x7f59317ba700>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:24.313917 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-f8a0fac3-d750-4df9-81b9-06b38fe49799 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] HTTP exception thrown: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f8a0fac3-d750-4df9-81b9-06b38fe49799 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Request method failure captured: Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: DELETE /compute/v2.1/os-volumes/93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f HTTP/1.1 Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZJgVUDWF113vWklHc3RJdVt8DJ5YCrk62yu2we2zhoIr7aroHG3AXLb2h8RF3djojz7w9cKsSG-Xko_jKZiXDi8fOLP4LWkealDsRoZhDOnxD1Lqrct4MlMnTIF0glAgCfNvtnoMOSrWPLZlmsT0lE1cm3nBpD509RW7WY44Wu4Rg Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: bf9d11f0b33a402f8584b4a158b3b41b Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-142704109 Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:40:24.316761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:40:24.318063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/bf9d11f0b33a402f8584b4a158b3b41b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/bf9d11f0b33a402f8584b4a158b3b41b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_bf9d11f0b33a402f8584b4a158b3b41b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/bf9d11f0b33a402f8584b4a158b3b41b"}]}] Jul 11 14:40:24.318063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-VolumesSnapshotsTestJSON-142704109 Jul 11 14:40:24.318063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: bf9d11f0b33a402f8584b4a158b3b41b Jul 11 14:40:24.318063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-142704109 Jul 11 14:40:24.318063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-VolumesSnapshotsTestJSON-142704109-project Jul 11 14:40:24.318063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:40:24.318063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:40:24.318063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 072255e7ae5147cb9a1540fd12f242da Jul 11 14:40:24.318063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-VolumesSnapshotsTestJSON-142704109-project Jul 11 14:40:24.318063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: Jul 11 14:40:24.318063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. Jul 11 14:40:24.318063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f'} Jul 11 14:40:24.318063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. (HTTP 404) (Request-ID: req-49d80ee1-3bc9-4587-b9b7-7ef5a18d17ee) Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:40:24.318888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 11 14:40:24.320034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 11 14:40:24.324446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:40:24.325676 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. Jul 11 14:40:24.325676 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.325676 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:40:24.325676 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.325676 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:40:24.325676 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:40:24.325676 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:40:24.325676 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:40:24.325676 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:40:24.325676 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. Jul 11 14:40:24.325676 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.325676 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f8a0fac3-d750-4df9-81b9-06b38fe49799 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Returning 404 to user: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:40:24.325676 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f8a0fac3-d750-4df9-81b9-06b38fe49799 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "DELETE /compute/v2.1/os-volumes/93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f" status: 404 len: 109 microversion: 2.1 time: 0.096928 Jul 11 14:40:24.325676 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 12/19] 10.222.0.72 () {58 vars in 1247 bytes} [Mon Jul 11 14:40:24 2022] DELETE /compute/v2.1/os-volumes/93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f => generated 109 bytes in 99 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:40:24.334541 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-5729f80f-bb8e-4229-97a9-c60ae79bc0e2 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Calling method '.version_select at 0x7f59317ba1f0>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:24.704503 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-5729f80f-bb8e-4229-97a9-c60ae79bc0e2 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] HTTP exception thrown: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-5729f80f-bb8e-4229-97a9-c60ae79bc0e2 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Request method failure captured: Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/os-volumes/93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f HTTP/1.1 Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZJgVUDWF113vWklHc3RJdVt8DJ5YCrk62yu2we2zhoIr7aroHG3AXLb2h8RF3djojz7w9cKsSG-Xko_jKZiXDi8fOLP4LWkealDsRoZhDOnxD1Lqrct4MlMnTIF0glAgCfNvtnoMOSrWPLZlmsT0lE1cm3nBpD509RW7WY44Wu4Rg Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: bf9d11f0b33a402f8584b4a158b3b41b Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-142704109 Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:40:24.706808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:40:24.707944 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/bf9d11f0b33a402f8584b4a158b3b41b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/bf9d11f0b33a402f8584b4a158b3b41b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_bf9d11f0b33a402f8584b4a158b3b41b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/bf9d11f0b33a402f8584b4a158b3b41b"}]}] Jul 11 14:40:24.707944 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-VolumesSnapshotsTestJSON-142704109 Jul 11 14:40:24.707944 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: bf9d11f0b33a402f8584b4a158b3b41b Jul 11 14:40:24.707944 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-142704109 Jul 11 14:40:24.707944 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-VolumesSnapshotsTestJSON-142704109-project Jul 11 14:40:24.707944 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:40:24.707944 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:40:24.707944 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 072255e7ae5147cb9a1540fd12f242da Jul 11 14:40:24.707944 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-VolumesSnapshotsTestJSON-142704109-project Jul 11 14:40:24.707944 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: Jul 11 14:40:24.707944 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. Jul 11 14:40:24.707944 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f'} Jul 11 14:40:24.707944 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. (HTTP 404) (Request-ID: req-c44090b9-6cc3-4e1e-ab28-b6214f560e19) Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:40:24.708770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 11 14:40:24.709660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 11 14:40:24.710628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:40:24.711516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. Jul 11 14:40:24.711516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.711516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:40:24.711516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.711516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:40:24.711516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:40:24.711516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:40:24.711516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:40:24.711516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:40:24.711516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. Jul 11 14:40:24.711516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:40:24.711516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-5729f80f-bb8e-4229-97a9-c60ae79bc0e2 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] Returning 404 to user: Volume 93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:40:24.711516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-5729f80f-bb8e-4229-97a9-c60ae79bc0e2 tempest-VolumesSnapshotsTestJSON-142704109 tempest-VolumesSnapshotsTestJSON-142704109-project] 10.222.0.72 "GET /compute/v2.1/os-volumes/93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f" status: 404 len: 109 microversion: 2.1 time: 0.383468 Jul 11 14:40:24.711516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 13/20] 10.222.0.72 () {58 vars in 1244 bytes} [Mon Jul 11 14:40:24 2022] GET /compute/v2.1/os-volumes/93d204d5-94d7-43e3-ae9f-8f34c9c2ae7f => generated 109 bytes in 385 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:40:25.174097 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-f37911d3-f5d9-4cbf-8eaf-a10ef2ec8390 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Flavor limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:40:25.174850 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-f37911d3-f5d9-4cbf-8eaf-a10ef2ec8390 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Image limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:40:25.175391 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-f37911d3-f5d9-4cbf-8eaf-a10ef2ec8390 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Flavor pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:40:25.175830 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-f37911d3-f5d9-4cbf-8eaf-a10ef2ec8390 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Image pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:40:25.176640 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-f37911d3-f5d9-4cbf-8eaf-a10ef2ec8390 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:40:25.242532 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f37911d3-f5d9-4cbf-8eaf-a10ef2ec8390 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:25.242994 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f37911d3-f5d9-4cbf-8eaf-a10ef2ec8390 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:25.243801 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f37911d3-f5d9-4cbf-8eaf-a10ef2ec8390 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:25.259574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f37911d3-f5d9-4cbf-8eaf-a10ef2ec8390 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:25.355109 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-f37911d3-f5d9-4cbf-8eaf-a10ef2ec8390 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Getting quotas for project 98966d44ea88477b94215761cdc92ea7. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:40:25.372653 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-f37911d3-f5d9-4cbf-8eaf-a10ef2ec8390 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Getting quotas for user fa478fb4b41e49c5b1f9291f925cca23 and project 98966d44ea88477b94215761cdc92ea7. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:40:25.443366 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f37911d3-f5d9-4cbf-8eaf-a10ef2ec8390 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Going to run 1 instances... {{(pid=130160) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:40:25.453980 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f37911d3-f5d9-4cbf-8eaf-a10ef2ec8390 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a588fa24-23b5-4e53-87c6-7e600ee3a5ef',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130160) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:40:25.592415 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f37911d3-f5d9-4cbf-8eaf-a10ef2ec8390 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 3.435400 Jul 11 14:40:25.594265 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 8/21] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:40:22 2022] POST /compute/v2.1/servers => generated 383 bytes in 3438 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 11 14:40:25.605204 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f78d0297-0ded-4b71-a5c9-d5c5feab0f06 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:25.606627 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f78d0297-0ded-4b71-a5c9-d5c5feab0f06 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:26.162416 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f78d0297-0ded-4b71-a5c9-d5c5feab0f06 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:26.191632 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f78d0297-0ded-4b71-a5c9-d5c5feab0f06 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1222 microversion: 2.1 time: 0.589556 Jul 11 14:40:26.195926 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 14/22] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:25 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1222 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:26.483704 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-ab62770b-1da9-4ca1-9d9f-9c809529eeb0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1976149094", "imageRef": "a588fa24-23b5-4e53-87c6-7e600ee3a5ef", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:40:26.683699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.network.neutron [None req-ab62770b-1da9-4ca1-9d9f-9c809529eeb0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] validate_networks() for [('382f35eb-d1cd-45e3-b28b-3e3389f544db', None, None, None, None, None)] {{(pid=130160) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:40:27.249701 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-56e7ed9e-5ebb-45ab-aa70-fad4a4752538 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Action: 'create', calling method: .version_select at 0x7f5931778820>, body: {"keypair": {"name": "tempest-keypair-258714009", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDP6PUGlIhUjXGF2uBHGn3Y1M5t3iB2xSJt/8Z/h+FswrVH66vnKw/bdw2F5HukUx0Tj1THcghwdRaGVAT6Uz3qiZWn56ew7zG14bb9yH/hoszLdHRg2x1XvDhjt7GGhHg=="}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:40:27.342098 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-56e7ed9e-5ebb-45ab-aa70-fad4a4752538 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Getting quotas for project f6606b37accd43619058aef23f10745b. Resources: {'key_pairs'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:40:27.355854 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-56e7ed9e-5ebb-45ab-aa70-fad4a4752538 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Getting quotas for user 99a62e49e6d249c2a169ef7122e762ec and project f6606b37accd43619058aef23f10745b. Resources: {'key_pairs'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:40:27.483629 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-56e7ed9e-5ebb-45ab-aa70-fad4a4752538 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.605205 Jul 11 14:40:27.485924 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 15/23] 10.222.0.72 () {62 vars in 1199 bytes} [Mon Jul 11 14:40:26 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 612 msecs (HTTP/1.1 201) 9 headers in 364 bytes (2 switches on core 0) Jul 11 14:40:27.967195 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-5c3cea1d-a159-4ce6-bcd7-e01b94994724 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Action: 'create', calling method: .version_select at 0x7f5930e831f0>, body: {"keypair": {"name": "tempest-keypair-1674144846", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPneOP3gTpK4DsqFkau7M6CmQNqTx8nhJGVpsKNZwI1AVEXiKcOl+7i60kCVLLcqUwZVmCaqes14XCINzf8WEMqSpc34xJaJcCai4axN9Wonc0Yseuf2d1aa7xzec18nbQ=="}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:40:28.018650 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-5c3cea1d-a159-4ce6-bcd7-e01b94994724 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Getting quotas for project 1baa6ea2e19747d1af1e1042c362895d. Resources: {'key_pairs'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:40:28.023242 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-5c3cea1d-a159-4ce6-bcd7-e01b94994724 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Getting quotas for user 3ae56fd1172f4bf8806eefc6e8877fef and project 1baa6ea2e19747d1af1e1042c362895d. Resources: {'key_pairs'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:40:28.080942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-5c3cea1d-a159-4ce6-bcd7-e01b94994724 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.229581 Jul 11 14:40:28.083543 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 16/24] 10.222.0.72 () {60 vars in 1158 bytes} [Mon Jul 11 14:40:27 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:40:28.472036 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-ab62770b-1da9-4ca1-9d9f-9c809529eeb0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Flavor limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:40:28.473128 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-ab62770b-1da9-4ca1-9d9f-9c809529eeb0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Image limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:40:28.473790 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-ab62770b-1da9-4ca1-9d9f-9c809529eeb0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Flavor pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:40:28.474337 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-ab62770b-1da9-4ca1-9d9f-9c809529eeb0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Image pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:40:28.475139 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-ab62770b-1da9-4ca1-9d9f-9c809529eeb0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:40:28.508080 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ab62770b-1da9-4ca1-9d9f-9c809529eeb0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:28.510658 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ab62770b-1da9-4ca1-9d9f-9c809529eeb0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:28.516723 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ab62770b-1da9-4ca1-9d9f-9c809529eeb0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:28.519491 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ab62770b-1da9-4ca1-9d9f-9c809529eeb0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:28.610642 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-ab62770b-1da9-4ca1-9d9f-9c809529eeb0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Getting quotas for project 98966d44ea88477b94215761cdc92ea7. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:40:28.631324 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-ab62770b-1da9-4ca1-9d9f-9c809529eeb0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Getting quotas for user fa478fb4b41e49c5b1f9291f925cca23 and project 98966d44ea88477b94215761cdc92ea7. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:40:28.654566 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-ab62770b-1da9-4ca1-9d9f-9c809529eeb0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Going to run 1 instances... {{(pid=130160) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:40:28.659693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-ab62770b-1da9-4ca1-9d9f-9c809529eeb0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a588fa24-23b5-4e53-87c6-7e600ee3a5ef',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130160) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:40:28.776437 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-ab62770b-1da9-4ca1-9d9f-9c809529eeb0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 2.298201 Jul 11 14:40:28.779340 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 9/25] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:40:26 2022] POST /compute/v2.1/servers => generated 383 bytes in 2302 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 11 14:40:28.788818 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9aff13f2-f827-4eb2-819b-79711719a171 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:28.790358 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9aff13f2-f827-4eb2-819b-79711719a171 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:29.364314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9aff13f2-f827-4eb2-819b-79711719a171 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:29.375937 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9aff13f2-f827-4eb2-819b-79711719a171 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1222 microversion: 2.1 time: 0.589652 Jul 11 14:40:29.377293 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 17/26] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:28 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1222 bytes in 593 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:30.401082 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-545453e4-ea71-4887-98b3-7bb4a39f95f1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:30.403017 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-545453e4-ea71-4887-98b3-7bb4a39f95f1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:31.179710 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-545453e4-ea71-4887-98b3-7bb4a39f95f1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:31.226352 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-545453e4-ea71-4887-98b3-7bb4a39f95f1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1222 microversion: 2.1 time: 0.834450 Jul 11 14:40:31.232549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 10/27] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:30 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1222 bytes in 843 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:32.248706 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-4d1782b8-b6a1-4991-a287-96e7982146e9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:32.249451 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-4d1782b8-b6a1-4991-a287-96e7982146e9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:32.261120 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4d1782b8-b6a1-4991-a287-96e7982146e9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:32.262152 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4d1782b8-b6a1-4991-a287-96e7982146e9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:32.263023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4d1782b8-b6a1-4991-a287-96e7982146e9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:32.263476 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4d1782b8-b6a1-4991-a287-96e7982146e9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:32.691818 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-bdc52828-532a-4b26-814c-9e633a1036bd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1476875986", "imageRef": "a588fa24-23b5-4e53-87c6-7e600ee3a5ef", "flavorRef": "452", "networks": [{"uuid": "5ae38dfd-9d35-4028-a44d-a35e6f13d941"}]}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:40:32.880550 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.network.neutron [None req-bdc52828-532a-4b26-814c-9e633a1036bd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] validate_networks() for [('5ae38dfd-9d35-4028-a44d-a35e6f13d941', None, None, None, None, None)] {{(pid=130160) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:40:33.006021 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-4d1782b8-b6a1-4991-a287-96e7982146e9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:33.030738 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-4d1782b8-b6a1-4991-a287-96e7982146e9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1282 microversion: 2.1 time: 0.784522 Jul 11 14:40:33.033880 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 18/28] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:32 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1282 bytes in 790 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:33.371946 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8a0c5ba5-c459-464d-a820-4e3f64551dbd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Action: 'create', calling method: .version_select at 0x7f593181f310>, body: {"keypair": {"name": "tempest-keypair-1843444996", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOCTLmYOQDIO3UlCBxKjR4wD5qDwJg6qDZ8gPlfwLj0Wj6a3VVM+m+C4Jfnhe5Y2n8oj00dp6EM96r1gC/qpqWQnVmASUHbCreN4o4gwdPkNHlx+N1RuIr1FbWCy//2JaQ=="}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:40:33.417292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-8a0c5ba5-c459-464d-a820-4e3f64551dbd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Getting quotas for project c0e363b1584848c1aecbe186cf91eec5. Resources: {'key_pairs'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:40:33.443508 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-8a0c5ba5-c459-464d-a820-4e3f64551dbd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Getting quotas for user 467992b53c4749858fd8b0ccd2db2393 and project c0e363b1584848c1aecbe186cf91eec5. Resources: {'key_pairs'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:40:33.533661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8a0c5ba5-c459-464d-a820-4e3f64551dbd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.334920 Jul 11 14:40:33.537881 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 19/29] 10.222.0.72 () {60 vars in 1158 bytes} [Mon Jul 11 14:40:33 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:40:34.056052 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-85e36d08-9980-4ffc-bd02-d68bdd482c4c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:34.057134 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-85e36d08-9980-4ffc-bd02-d68bdd482c4c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:34.087438 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-85e36d08-9980-4ffc-bd02-d68bdd482c4c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:34.089098 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-85e36d08-9980-4ffc-bd02-d68bdd482c4c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:34.095520 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-85e36d08-9980-4ffc-bd02-d68bdd482c4c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:34.098630 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-85e36d08-9980-4ffc-bd02-d68bdd482c4c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:34.780604 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-85e36d08-9980-4ffc-bd02-d68bdd482c4c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:34.804530 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-85e36d08-9980-4ffc-bd02-d68bdd482c4c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1280 microversion: 2.1 time: 0.751367 Jul 11 14:40:34.814004 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 20/30] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:34 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1280 bytes in 769 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:35.837778 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d5278ffb-261a-45bd-b0c1-8e07217e3999 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:35.839268 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d5278ffb-261a-45bd-b0c1-8e07217e3999 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:35.850818 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d5278ffb-261a-45bd-b0c1-8e07217e3999 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:35.853125 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d5278ffb-261a-45bd-b0c1-8e07217e3999 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:35.857739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d5278ffb-261a-45bd-b0c1-8e07217e3999 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:35.861513 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d5278ffb-261a-45bd-b0c1-8e07217e3999 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:36.721980 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d5278ffb-261a-45bd-b0c1-8e07217e3999 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:36.731109 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-d5278ffb-261a-45bd-b0c1-8e07217e3999 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1280 microversion: 2.1 time: 0.903222 Jul 11 14:40:36.734069 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 21/31] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:35 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1280 bytes in 906 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:36.766303 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-bdc52828-532a-4b26-814c-9e633a1036bd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Flavor limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:40:36.767740 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-bdc52828-532a-4b26-814c-9e633a1036bd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Image limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:40:36.769773 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-bdc52828-532a-4b26-814c-9e633a1036bd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Flavor pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:40:36.771872 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-bdc52828-532a-4b26-814c-9e633a1036bd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Image pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:40:36.772714 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-bdc52828-532a-4b26-814c-9e633a1036bd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:40:36.848063 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bdc52828-532a-4b26-814c-9e633a1036bd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:36.849113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bdc52828-532a-4b26-814c-9e633a1036bd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:36.852799 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bdc52828-532a-4b26-814c-9e633a1036bd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:36.854237 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bdc52828-532a-4b26-814c-9e633a1036bd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:36.990883 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-bdc52828-532a-4b26-814c-9e633a1036bd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Getting quotas for project fd0be78cdcfc4195a0ced5868c14cda5. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:40:36.999037 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-bdc52828-532a-4b26-814c-9e633a1036bd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Getting quotas for user 85705ea8eb6b43e18d14eb5efefab96f and project fd0be78cdcfc4195a0ced5868c14cda5. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:40:37.040411 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-bdc52828-532a-4b26-814c-9e633a1036bd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Going to run 1 instances... {{(pid=130160) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:40:37.047419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-bdc52828-532a-4b26-814c-9e633a1036bd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a588fa24-23b5-4e53-87c6-7e600ee3a5ef',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130160) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:40:37.118860 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-bdc52828-532a-4b26-814c-9e633a1036bd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 4.737815 Jul 11 14:40:37.121258 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 11/32] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:40:32 2022] POST /compute/v2.1/servers => generated 380 bytes in 4741 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 11 14:40:37.131392 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1e964852-ee90-408a-a576-046b214e4478 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:37.132864 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1e964852-ee90-408a-a576-046b214e4478 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:37.547236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1e964852-ee90-408a-a576-046b214e4478 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:37.555098 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1e964852-ee90-408a-a576-046b214e4478 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1215 microversion: 2.1 time: 0.427383 Jul 11 14:40:37.557433 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 22/33] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:37 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1215 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:37.751085 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-eaaa98a1-9e2f-44f3-bfd6-c06076f63159 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:37.752019 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-eaaa98a1-9e2f-44f3-bfd6-c06076f63159 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:37.772674 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eaaa98a1-9e2f-44f3-bfd6-c06076f63159 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:37.775886 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eaaa98a1-9e2f-44f3-bfd6-c06076f63159 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:37.778736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eaaa98a1-9e2f-44f3-bfd6-c06076f63159 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:37.780625 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eaaa98a1-9e2f-44f3-bfd6-c06076f63159 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:38.573348 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9606e1e1-ae32-488c-963c-7a5c458f00f8 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:38.574831 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9606e1e1-ae32-488c-963c-7a5c458f00f8 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:38.600742 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9606e1e1-ae32-488c-963c-7a5c458f00f8 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:38.601557 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9606e1e1-ae32-488c-963c-7a5c458f00f8 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:38.602729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9606e1e1-ae32-488c-963c-7a5c458f00f8 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:38.603489 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9606e1e1-ae32-488c-963c-7a5c458f00f8 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:38.800339 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9606e1e1-ae32-488c-963c-7a5c458f00f8 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:38.808760 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9606e1e1-ae32-488c-963c-7a5c458f00f8 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1219 microversion: 2.1 time: 0.239493 Jul 11 14:40:38.809358 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 23/34] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:38 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1219 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:40:39.413631 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-eaaa98a1-9e2f-44f3-bfd6-c06076f63159 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:39.429831 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c83f9dc1-5236-4a7c-aed8-94ed2a0f7fed tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1325034955", "imageRef": "a588fa24-23b5-4e53-87c6-7e600ee3a5ef", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8679b039-4e4e-4ae1-83e8-21c78c675a96"}], "security_groups": [{"name": "tempest-securitygroup--2075129506"}], "key_name": "tempest-keypair-1674144846", "user_data": "}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:40:39.453501 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-eaaa98a1-9e2f-44f3-bfd6-c06076f63159 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1322 microversion: 2.1 time: 1.707548 Jul 11 14:40:39.454108 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 12/35] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:37 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1322 bytes in 1709 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:39.828682 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-4b426221-5061-4f45-a5a7-a29639fba736 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:39.829795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-4b426221-5061-4f45-a5a7-a29639fba736 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:39.846322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4b426221-5061-4f45-a5a7-a29639fba736 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:39.852545 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4b426221-5061-4f45-a5a7-a29639fba736 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:39.856291 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4b426221-5061-4f45-a5a7-a29639fba736 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:39.873309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4b426221-5061-4f45-a5a7-a29639fba736 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:40.359927 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4b426221-5061-4f45-a5a7-a29639fba736 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:40.375467 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-4b426221-5061-4f45-a5a7-a29639fba736 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1211 microversion: 2.1 time: 0.553515 Jul 11 14:40:40.387649 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 13/36] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:39 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1211 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:40.471596 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-5b82b5b6-55c0-4d3e-98c1-4681873369f4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:40.472299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-5b82b5b6-55c0-4d3e-98c1-4681873369f4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:40.486188 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5b82b5b6-55c0-4d3e-98c1-4681873369f4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:40.486686 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5b82b5b6-55c0-4d3e-98c1-4681873369f4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:40.490895 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5b82b5b6-55c0-4d3e-98c1-4681873369f4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:40.496637 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5b82b5b6-55c0-4d3e-98c1-4681873369f4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:40.694179 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.network.neutron [None req-c83f9dc1-5236-4a7c-aed8-94ed2a0f7fed tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] validate_networks() for [('8679b039-4e4e-4ae1-83e8-21c78c675a96', None, None, None, None, None)] {{(pid=130159) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:40:41.215613 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5b82b5b6-55c0-4d3e-98c1-4681873369f4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:41.224292 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-5b82b5b6-55c0-4d3e-98c1-4681873369f4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1322 microversion: 2.1 time: 0.757016 Jul 11 14:40:41.224968 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 14/37] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:40 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1322 bytes in 759 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:41.401443 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-5fbe1e9d-eeaf-41e8-b8a2-711d1e00d92a tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:41.402112 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-5fbe1e9d-eeaf-41e8-b8a2-711d1e00d92a tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:41.425643 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5fbe1e9d-eeaf-41e8-b8a2-711d1e00d92a tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:41.434320 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5fbe1e9d-eeaf-41e8-b8a2-711d1e00d92a tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:41.440048 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5fbe1e9d-eeaf-41e8-b8a2-711d1e00d92a tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:41.441224 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5fbe1e9d-eeaf-41e8-b8a2-711d1e00d92a tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:41.689224 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5fbe1e9d-eeaf-41e8-b8a2-711d1e00d92a tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:41.703583 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-5fbe1e9d-eeaf-41e8-b8a2-711d1e00d92a tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1211 microversion: 2.1 time: 0.313435 Jul 11 14:40:41.714685 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 15/38] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:41 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1211 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:42.379372 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-7b75b9f8-7eb5-46c5-8cd9-5b5a56fcc130 req-e0906cbf-451a-4775-b539-c5fcf6b555dc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a0498735-9e0a-4f14-990d-a6d5076e742a", "tag": "27aef129-6bd8-4686-9e6d-915b57154c5a"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:40:42.416833 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-7b75b9f8-7eb5-46c5-8cd9-5b5a56fcc130 req-e0906cbf-451a-4775-b539-c5fcf6b555dc service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:42.417969 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-7b75b9f8-7eb5-46c5-8cd9-5b5a56fcc130 req-e0906cbf-451a-4775-b539-c5fcf6b555dc service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:42.443952 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-7b75b9f8-7eb5-46c5-8cd9-5b5a56fcc130 req-e0906cbf-451a-4775-b539-c5fcf6b555dc service nova] Creating event network-changed:27aef129-6bd8-4686-9e6d-915b57154c5a for instance a0498735-9e0a-4f14-990d-a6d5076e742a on hv-cinder-maste Jul 11 14:40:42.445364 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-7b75b9f8-7eb5-46c5-8cd9-5b5a56fcc130 req-e0906cbf-451a-4775-b539-c5fcf6b555dc service nova] Acquired lock "compute-rpcapi-router" {{(pid=130160) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 11 14:40:42.446843 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8ee04f03-1b38-4208-a8e8-e9fbb20e41a0 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:42.447526 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8ee04f03-1b38-4208-a8e8-e9fbb20e41a0 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:42.448573 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8ee04f03-1b38-4208-a8e8-e9fbb20e41a0 None None] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:42.449249 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8ee04f03-1b38-4208-a8e8-e9fbb20e41a0 None None] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:42.461697 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.compute.rpcapi [None req-8ee04f03-1b38-4208-a8e8-e9fbb20e41a0 None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Jul 11 14:40:42.462806 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8ee04f03-1b38-4208-a8e8-e9fbb20e41a0 None None] Releasing lock "compute-rpcapi-router" {{(pid=130160) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 11 14:40:42.508804 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8ee04f03-1b38-4208-a8e8-e9fbb20e41a0 None None] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.271233 Jul 11 14:40:42.513985 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 16/39] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:40:42 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:40:42.532372 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-045fe08e-51ad-4056-9189-7614aeda9fb2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:42.543274 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-045fe08e-51ad-4056-9189-7614aeda9fb2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:42.569673 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-045fe08e-51ad-4056-9189-7614aeda9fb2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:42.578825 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-045fe08e-51ad-4056-9189-7614aeda9fb2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:42.580320 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-045fe08e-51ad-4056-9189-7614aeda9fb2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:42.581008 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-045fe08e-51ad-4056-9189-7614aeda9fb2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:42.904128 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-045fe08e-51ad-4056-9189-7614aeda9fb2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:42.915091 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-045fe08e-51ad-4056-9189-7614aeda9fb2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1322 microversion: 2.1 time: 0.399342 Jul 11 14:40:42.915941 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 17/40] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:42 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1322 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:42.936827 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-fd725866-7016-4864-9b46-6349fcdd61ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1383290771", "imageRef": "a588fa24-23b5-4e53-87c6-7e600ee3a5ef", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "164cf2a3-5444-4c03-8807-160bd45f2610"}], "security_groups": [{"name": "tempest-securitygroup--1911990637"}], "key_name": "tempest-keypair-258714009", "user_data": "}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:40:43.113644 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-c83f9dc1-5236-4a7c-aed8-94ed2a0f7fed tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Flavor limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:40:43.114500 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-c83f9dc1-5236-4a7c-aed8-94ed2a0f7fed tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Image limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:40:43.115149 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-c83f9dc1-5236-4a7c-aed8-94ed2a0f7fed tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Flavor pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:40:43.116312 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-c83f9dc1-5236-4a7c-aed8-94ed2a0f7fed tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Image pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:40:43.116765 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-c83f9dc1-5236-4a7c-aed8-94ed2a0f7fed tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:40:43.138234 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c83f9dc1-5236-4a7c-aed8-94ed2a0f7fed tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:43.142782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c83f9dc1-5236-4a7c-aed8-94ed2a0f7fed tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:43.144435 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c83f9dc1-5236-4a7c-aed8-94ed2a0f7fed tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:43.147247 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c83f9dc1-5236-4a7c-aed8-94ed2a0f7fed tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:43.225758 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-c83f9dc1-5236-4a7c-aed8-94ed2a0f7fed tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Getting quotas for project 1baa6ea2e19747d1af1e1042c362895d. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:40:43.228008 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.network.neutron [None req-fd725866-7016-4864-9b46-6349fcdd61ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] validate_networks() for [('164cf2a3-5444-4c03-8807-160bd45f2610', None, None, None, None, None)] {{(pid=130160) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:40:43.249871 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-c83f9dc1-5236-4a7c-aed8-94ed2a0f7fed tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Getting quotas for user 3ae56fd1172f4bf8806eefc6e8877fef and project 1baa6ea2e19747d1af1e1042c362895d. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:40:43.317436 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c83f9dc1-5236-4a7c-aed8-94ed2a0f7fed tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Going to run 1 instances... {{(pid=130159) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:40:43.333555 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c83f9dc1-5236-4a7c-aed8-94ed2a0f7fed tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a588fa24-23b5-4e53-87c6-7e600ee3a5ef',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130159) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:40:43.560691 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c83f9dc1-5236-4a7c-aed8-94ed2a0f7fed tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 4.138437 Jul 11 14:40:43.564014 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 24/41] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:40:39 2022] POST /compute/v2.1/servers => generated 402 bytes in 4142 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 11 14:40:43.581092 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d01e8ac2-b2c0-4254-9c84-4eacbf9bb4ce tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:43.581949 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d01e8ac2-b2c0-4254-9c84-4eacbf9bb4ce tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:43.599191 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d01e8ac2-b2c0-4254-9c84-4eacbf9bb4ce tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:43.602021 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d01e8ac2-b2c0-4254-9c84-4eacbf9bb4ce tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:43.603799 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d01e8ac2-b2c0-4254-9c84-4eacbf9bb4ce tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:43.606080 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d01e8ac2-b2c0-4254-9c84-4eacbf9bb4ce tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:43.872654 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d01e8ac2-b2c0-4254-9c84-4eacbf9bb4ce tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:43.905372 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-d01e8ac2-b2c0-4254-9c84-4eacbf9bb4ce tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1267 microversion: 2.1 time: 0.339215 Jul 11 14:40:43.918072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 25/42] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:43 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1267 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:44.091096 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-7597c588-458b-470d-9556-88ce0975feec req-0c232f06-965b-4c6d-8c2a-224e8a6f4856 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cc0591e3-09b9-41cd-b222-f730282f8572", "tag": "9af04c14-ce8d-46f1-8097-0f76fe117ff9"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:40:44.124493 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-7597c588-458b-470d-9556-88ce0975feec req-0c232f06-965b-4c6d-8c2a-224e8a6f4856 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:44.126118 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-7597c588-458b-470d-9556-88ce0975feec req-0c232f06-965b-4c6d-8c2a-224e8a6f4856 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:44.193932 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-7597c588-458b-470d-9556-88ce0975feec req-0c232f06-965b-4c6d-8c2a-224e8a6f4856 service nova] Creating event network-changed:9af04c14-ce8d-46f1-8097-0f76fe117ff9 for instance cc0591e3-09b9-41cd-b222-f730282f8572 on hv-cinder-maste Jul 11 14:40:44.199471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-7597c588-458b-470d-9556-88ce0975feec req-0c232f06-965b-4c6d-8c2a-224e8a6f4856 service nova] Acquired lock "compute-rpcapi-router" {{(pid=130159) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} Jul 11 14:40:44.201253 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b6126c3e-adde-41c8-9ba1-b1a670123fae None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:44.201712 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b6126c3e-adde-41c8-9ba1-b1a670123fae None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:44.207799 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b6126c3e-adde-41c8-9ba1-b1a670123fae None None] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:44.208673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b6126c3e-adde-41c8-9ba1-b1a670123fae None None] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:44.229050 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.compute.rpcapi [None req-b6126c3e-adde-41c8-9ba1-b1a670123fae None None] Automatically selected compute RPC version 6.0 from minimum service version 61 Jul 11 14:40:44.271335 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b6126c3e-adde-41c8-9ba1-b1a670123fae None None] Releasing lock "compute-rpcapi-router" {{(pid=130159) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Jul 11 14:40:44.297291 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-b6126c3e-adde-41c8-9ba1-b1a670123fae None None] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.377194 Jul 11 14:40:44.312784 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 26/43] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:40:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:40:44.336640 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2132e5de-611c-49cf-827d-3958b7214d8c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:44.338420 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2132e5de-611c-49cf-827d-3958b7214d8c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:44.437962 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-fd725866-7016-4864-9b46-6349fcdd61ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Flavor limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:40:44.438683 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-fd725866-7016-4864-9b46-6349fcdd61ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Image limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:40:44.439745 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-fd725866-7016-4864-9b46-6349fcdd61ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Flavor pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:40:44.440357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-fd725866-7016-4864-9b46-6349fcdd61ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Image pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:40:44.441141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-fd725866-7016-4864-9b46-6349fcdd61ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:40:44.455990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd725866-7016-4864-9b46-6349fcdd61ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:44.456935 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd725866-7016-4864-9b46-6349fcdd61ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:44.458869 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd725866-7016-4864-9b46-6349fcdd61ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:44.459628 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd725866-7016-4864-9b46-6349fcdd61ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:44.475493 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-2132e5de-611c-49cf-827d-3958b7214d8c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:44.483433 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2132e5de-611c-49cf-827d-3958b7214d8c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1242 microversion: 2.1 time: 0.165141 Jul 11 14:40:44.487597 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 27/44] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:44 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1242 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:44.503345 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-fd725866-7016-4864-9b46-6349fcdd61ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Getting quotas for project f6606b37accd43619058aef23f10745b. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:40:44.509682 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-fd725866-7016-4864-9b46-6349fcdd61ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Getting quotas for user 99a62e49e6d249c2a169ef7122e762ec and project f6606b37accd43619058aef23f10745b. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:40:44.512543 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-4d5a24b8-a2eb-47d7-b430-5067f3007384 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:44.516079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-4d5a24b8-a2eb-47d7-b430-5067f3007384 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:44.528184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-fd725866-7016-4864-9b46-6349fcdd61ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Going to run 1 instances... {{(pid=130160) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:40:44.530695 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-fd725866-7016-4864-9b46-6349fcdd61ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a588fa24-23b5-4e53-87c6-7e600ee3a5ef',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130160) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:40:44.532502 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4d5a24b8-a2eb-47d7-b430-5067f3007384 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:44.532960 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4d5a24b8-a2eb-47d7-b430-5067f3007384 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:44.533747 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4d5a24b8-a2eb-47d7-b430-5067f3007384 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:44.538107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4d5a24b8-a2eb-47d7-b430-5067f3007384 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:44.599027 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-fd725866-7016-4864-9b46-6349fcdd61ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 1.680245 Jul 11 14:40:44.600531 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 18/45] 10.222.0.72 () {62 vars in 1187 bytes} [Mon Jul 11 14:40:42 2022] POST /compute/v2.1/servers => generated 402 bytes in 1684 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 11 14:40:44.612352 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-5192227b-b9fd-4f2c-9e5e-84400221e64d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-550399731", "imageRef": "a588fa24-23b5-4e53-87c6-7e600ee3a5ef", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "208c5447-9d23-4b47-b27a-4307a7296819"}], "security_groups": [{"name": "tempest-securitygroup--77027763"}], "key_name": "tempest-keypair-1843444996", "user_data": "}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:40:44.737736 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-4d5a24b8-a2eb-47d7-b430-5067f3007384 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:44.745306 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-4d5a24b8-a2eb-47d7-b430-5067f3007384 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1322 microversion: 2.1 time: 0.245191 Jul 11 14:40:44.746016 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 28/46] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:44 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1322 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:44.759410 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e5cbaace-f286-4b4a-a5fa-26c309c4aabf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:44.760377 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e5cbaace-f286-4b4a-a5fa-26c309c4aabf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:44.779628 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.network.neutron [None req-5192227b-b9fd-4f2c-9e5e-84400221e64d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] validate_networks() for [('208c5447-9d23-4b47-b27a-4307a7296819', None, None, None, None, None)] {{(pid=130160) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:40:44.818709 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e5cbaace-f286-4b4a-a5fa-26c309c4aabf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:44.828393 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e5cbaace-f286-4b4a-a5fa-26c309c4aabf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:44.830457 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e5cbaace-f286-4b4a-a5fa-26c309c4aabf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:44.831608 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e5cbaace-f286-4b4a-a5fa-26c309c4aabf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1281 microversion: 2.20 time: 0.076097 Jul 11 14:40:44.840945 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 29/47] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:40:44 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1281 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:40:44.925169 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-19c5a227-3d99-443a-8f1f-50c76efd0752 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:44.926004 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-19c5a227-3d99-443a-8f1f-50c76efd0752 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:44.933892 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-19c5a227-3d99-443a-8f1f-50c76efd0752 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:44.934821 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-19c5a227-3d99-443a-8f1f-50c76efd0752 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:44.935801 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-19c5a227-3d99-443a-8f1f-50c76efd0752 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:44.936567 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-19c5a227-3d99-443a-8f1f-50c76efd0752 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:45.005824 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-19c5a227-3d99-443a-8f1f-50c76efd0752 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:45.016033 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-19c5a227-3d99-443a-8f1f-50c76efd0752 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1267 microversion: 2.1 time: 0.095353 Jul 11 14:40:45.017258 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 30/48] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:44 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1267 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:40:45.294544 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-5192227b-b9fd-4f2c-9e5e-84400221e64d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Flavor limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:40:45.295290 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-5192227b-b9fd-4f2c-9e5e-84400221e64d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Image limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:40:45.296002 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-5192227b-b9fd-4f2c-9e5e-84400221e64d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Flavor pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:40:45.296400 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-5192227b-b9fd-4f2c-9e5e-84400221e64d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Image pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:40:45.297753 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-5192227b-b9fd-4f2c-9e5e-84400221e64d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:40:45.309077 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5192227b-b9fd-4f2c-9e5e-84400221e64d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:45.309737 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5192227b-b9fd-4f2c-9e5e-84400221e64d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:45.310754 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5192227b-b9fd-4f2c-9e5e-84400221e64d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:45.311306 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5192227b-b9fd-4f2c-9e5e-84400221e64d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:45.335453 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-5192227b-b9fd-4f2c-9e5e-84400221e64d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Getting quotas for project c0e363b1584848c1aecbe186cf91eec5. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:40:45.340485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-5192227b-b9fd-4f2c-9e5e-84400221e64d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Getting quotas for user 467992b53c4749858fd8b0ccd2db2393 and project c0e363b1584848c1aecbe186cf91eec5. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:40:45.356016 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-5192227b-b9fd-4f2c-9e5e-84400221e64d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Going to run 1 instances... {{(pid=130160) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:40:45.358323 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-5192227b-b9fd-4f2c-9e5e-84400221e64d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a588fa24-23b5-4e53-87c6-7e600ee3a5ef',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130160) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:40:45.393987 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=85,nova_api:INSERT=15 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:40:45.418203 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-5192227b-b9fd-4f2c-9e5e-84400221e64d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.1 time: 0.816078 Jul 11 14:40:45.419360 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 19/49] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:40:44 2022] POST /compute/v2.1/servers => generated 400 bytes in 818 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 11 14:40:45.429801 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-163b92c7-fe10-42a1-aa9c-8c2942b49f89 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:45.430736 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-163b92c7-fe10-42a1-aa9c-8c2942b49f89 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:45.514771 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3098d052-614a-4f71-92a2-cd4f4aeeaf59 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:45.515405 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-3098d052-614a-4f71-92a2-cd4f4aeeaf59 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:45.517010 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-163b92c7-fe10-42a1-aa9c-8c2942b49f89 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:45.539272 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3098d052-614a-4f71-92a2-cd4f4aeeaf59 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:45.540738 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3098d052-614a-4f71-92a2-cd4f4aeeaf59 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:45.545972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3098d052-614a-4f71-92a2-cd4f4aeeaf59 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:45.546853 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3098d052-614a-4f71-92a2-cd4f4aeeaf59 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:45.553839 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-163b92c7-fe10-42a1-aa9c-8c2942b49f89 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1237 microversion: 2.1 time: 0.128273 Jul 11 14:40:45.563997 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 31/50] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:45 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1237 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:40:45.674549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-3098d052-614a-4f71-92a2-cd4f4aeeaf59 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:45.693094 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-3098d052-614a-4f71-92a2-cd4f4aeeaf59 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1246 microversion: 2.1 time: 0.191108 Jul 11 14:40:45.693727 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 20/51] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:45 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1246 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:45.771270 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d9cc09e5-5889-44ce-aed8-99695bf31b7d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:45.772620 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d9cc09e5-5889-44ce-aed8-99695bf31b7d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:45.780761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d9cc09e5-5889-44ce-aed8-99695bf31b7d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:45.782301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d9cc09e5-5889-44ce-aed8-99695bf31b7d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:45.783858 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d9cc09e5-5889-44ce-aed8-99695bf31b7d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:45.784862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d9cc09e5-5889-44ce-aed8-99695bf31b7d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:45.858174 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-01b306ba-f840-456e-acf6-9c53373780c0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:45.860879 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-01b306ba-f840-456e-acf6-9c53373780c0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:45.873318 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-01b306ba-f840-456e-acf6-9c53373780c0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:45.874190 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-01b306ba-f840-456e-acf6-9c53373780c0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:45.876126 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-01b306ba-f840-456e-acf6-9c53373780c0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:45.877521 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-01b306ba-f840-456e-acf6-9c53373780c0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:46.105656 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-01b306ba-f840-456e-acf6-9c53373780c0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:46.118483 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-01b306ba-f840-456e-acf6-9c53373780c0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:46.120473 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-01b306ba-f840-456e-acf6-9c53373780c0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:46.121915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-01b306ba-f840-456e-acf6-9c53373780c0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1285 microversion: 2.20 time: 0.272602 Jul 11 14:40:46.123121 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 21/52] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:40:45 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1285 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:40:46.143805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-ffc8c8b8-e1e0-477f-9188-caae0a5d029d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:46.143805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-ffc8c8b8-e1e0-477f-9188-caae0a5d029d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:46.152247 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ffc8c8b8-e1e0-477f-9188-caae0a5d029d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:46.155988 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ffc8c8b8-e1e0-477f-9188-caae0a5d029d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:46.169548 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d9cc09e5-5889-44ce-aed8-99695bf31b7d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:46.180277 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-d9cc09e5-5889-44ce-aed8-99695bf31b7d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1322 microversion: 2.1 time: 0.421558 Jul 11 14:40:46.183653 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ffc8c8b8-e1e0-477f-9188-caae0a5d029d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:46.184513 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ffc8c8b8-e1e0-477f-9188-caae0a5d029d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:46.196805 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 32/53] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:45 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1322 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:46.539152 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-ffc8c8b8-e1e0-477f-9188-caae0a5d029d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:46.549065 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-ffc8c8b8-e1e0-477f-9188-caae0a5d029d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1267 microversion: 2.1 time: 0.424627 Jul 11 14:40:46.570229 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 22/54] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:46 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1267 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:46.574987 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-4d6ab56a-ed70-4600-9527-4c791715cc96 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:46.575613 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-4d6ab56a-ed70-4600-9527-4c791715cc96 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:46.585978 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4d6ab56a-ed70-4600-9527-4c791715cc96 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:46.586463 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4d6ab56a-ed70-4600-9527-4c791715cc96 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:46.587254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4d6ab56a-ed70-4600-9527-4c791715cc96 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:46.587697 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4d6ab56a-ed70-4600-9527-4c791715cc96 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:46.713171 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2df8cc7d-5d9b-4e63-917f-3ddb84befc2e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:46.714622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2df8cc7d-5d9b-4e63-917f-3ddb84befc2e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:46.736072 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2df8cc7d-5d9b-4e63-917f-3ddb84befc2e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:46.741983 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2df8cc7d-5d9b-4e63-917f-3ddb84befc2e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:46.746062 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2df8cc7d-5d9b-4e63-917f-3ddb84befc2e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:46.754949 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2df8cc7d-5d9b-4e63-917f-3ddb84befc2e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:46.766987 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-4d6ab56a-ed70-4600-9527-4c791715cc96 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:46.778413 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-4d6ab56a-ed70-4600-9527-4c791715cc96 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1241 microversion: 2.1 time: 0.209801 Jul 11 14:40:46.787770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 33/55] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:46 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1241 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:40:46.933126 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2df8cc7d-5d9b-4e63-917f-3ddb84befc2e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:46.962654 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2df8cc7d-5d9b-4e63-917f-3ddb84befc2e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1294 microversion: 2.1 time: 0.252203 Jul 11 14:40:46.969918 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 23/56] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:46 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1294 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:40:47.142380 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d6a3a29e-5ec3-450c-9029-d72eb7e3ae7a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:47.143448 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d6a3a29e-5ec3-450c-9029-d72eb7e3ae7a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:47.148469 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:40:47.188734 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d6a3a29e-5ec3-450c-9029-d72eb7e3ae7a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:47.191314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d6a3a29e-5ec3-450c-9029-d72eb7e3ae7a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:47.200348 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d6a3a29e-5ec3-450c-9029-d72eb7e3ae7a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:47.202942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d6a3a29e-5ec3-450c-9029-d72eb7e3ae7a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:47.205703 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c303bbf3-f5bf-44c2-b29c-faa892a2efc6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:47.206339 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c303bbf3-f5bf-44c2-b29c-faa892a2efc6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:47.222348 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c303bbf3-f5bf-44c2-b29c-faa892a2efc6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:47.226721 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c303bbf3-f5bf-44c2-b29c-faa892a2efc6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:47.235016 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c303bbf3-f5bf-44c2-b29c-faa892a2efc6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:47.239231 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c303bbf3-f5bf-44c2-b29c-faa892a2efc6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:47.444954 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d6a3a29e-5ec3-450c-9029-d72eb7e3ae7a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:47.463179 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d6a3a29e-5ec3-450c-9029-d72eb7e3ae7a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:47.467975 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d6a3a29e-5ec3-450c-9029-d72eb7e3ae7a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:47.474577 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-d6a3a29e-5ec3-450c-9029-d72eb7e3ae7a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1285 microversion: 2.20 time: 0.337057 Jul 11 14:40:47.476824 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 34/57] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:40:47 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1285 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:40:47.568665 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2896624d-223a-4593-a2fd-5ef9ded8c171 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:47.569586 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2896624d-223a-4593-a2fd-5ef9ded8c171 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:47.583817 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2896624d-223a-4593-a2fd-5ef9ded8c171 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:47.592159 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2896624d-223a-4593-a2fd-5ef9ded8c171 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:47.598153 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2896624d-223a-4593-a2fd-5ef9ded8c171 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:47.598966 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2896624d-223a-4593-a2fd-5ef9ded8c171 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:47.729263 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c303bbf3-f5bf-44c2-b29c-faa892a2efc6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:47.747945 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c303bbf3-f5bf-44c2-b29c-faa892a2efc6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1322 microversion: 2.1 time: 0.540621 Jul 11 14:40:47.748181 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-2896624d-223a-4593-a2fd-5ef9ded8c171 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:47.749995 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 24/58] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:47 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1322 bytes in 549 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:40:47.757400 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2896624d-223a-4593-a2fd-5ef9ded8c171 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1275 microversion: 2.1 time: 0.193617 Jul 11 14:40:47.762479 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 35/59] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:47 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1275 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:40:47.797643 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3b0dbbdd-74de-4853-bc16-29124e91c596 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:47.798270 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-3b0dbbdd-74de-4853-bc16-29124e91c596 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:47.813934 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3b0dbbdd-74de-4853-bc16-29124e91c596 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:47.814670 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3b0dbbdd-74de-4853-bc16-29124e91c596 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:47.815619 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3b0dbbdd-74de-4853-bc16-29124e91c596 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:47.816221 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3b0dbbdd-74de-4853-bc16-29124e91c596 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:47.954142 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-3b0dbbdd-74de-4853-bc16-29124e91c596 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:47.962955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-3b0dbbdd-74de-4853-bc16-29124e91c596 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1241 microversion: 2.1 time: 0.170726 Jul 11 14:40:47.983979 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 25/60] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:47 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1241 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:47.987335 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-36d4e6c6-f5f2-4d75-8711-cb9f7ec73ce9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:47.988037 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-36d4e6c6-f5f2-4d75-8711-cb9f7ec73ce9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:48.005310 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-36d4e6c6-f5f2-4d75-8711-cb9f7ec73ce9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:48.005809 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-36d4e6c6-f5f2-4d75-8711-cb9f7ec73ce9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:48.006678 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-36d4e6c6-f5f2-4d75-8711-cb9f7ec73ce9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:48.007161 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-36d4e6c6-f5f2-4d75-8711-cb9f7ec73ce9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:48.158389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-36d4e6c6-f5f2-4d75-8711-cb9f7ec73ce9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:48.186223 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-36d4e6c6-f5f2-4d75-8711-cb9f7ec73ce9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1294 microversion: 2.1 time: 0.202008 Jul 11 14:40:48.187075 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 36/61] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:47 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1294 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:48.498864 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d9a7dc5b-bd49-4bba-bad7-9ed2e8068226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:48.503228 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d9a7dc5b-bd49-4bba-bad7-9ed2e8068226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:48.516042 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d9a7dc5b-bd49-4bba-bad7-9ed2e8068226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:48.523977 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d9a7dc5b-bd49-4bba-bad7-9ed2e8068226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:48.524971 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d9a7dc5b-bd49-4bba-bad7-9ed2e8068226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:48.528875 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d9a7dc5b-bd49-4bba-bad7-9ed2e8068226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:48.601464 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d9a7dc5b-bd49-4bba-bad7-9ed2e8068226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:48.637743 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d9a7dc5b-bd49-4bba-bad7-9ed2e8068226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:48.645006 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d9a7dc5b-bd49-4bba-bad7-9ed2e8068226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:48.647207 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d9a7dc5b-bd49-4bba-bad7-9ed2e8068226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1277 microversion: 2.20 time: 0.158608 Jul 11 14:40:48.650780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 26/62] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:40:48 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1277 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:40:48.766119 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-fa8b228e-1bdb-431c-944e-29e7093a6a92 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:48.768524 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-fa8b228e-1bdb-431c-944e-29e7093a6a92 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:48.775116 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-492a5adb-5b70-4593-b651-c2969f419610 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:48.776978 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-492a5adb-5b70-4593-b651-c2969f419610 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:48.793789 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fa8b228e-1bdb-431c-944e-29e7093a6a92 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:48.796185 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-492a5adb-5b70-4593-b651-c2969f419610 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:48.798740 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-492a5adb-5b70-4593-b651-c2969f419610 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:48.800830 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-492a5adb-5b70-4593-b651-c2969f419610 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:48.802020 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fa8b228e-1bdb-431c-944e-29e7093a6a92 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:48.803118 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fa8b228e-1bdb-431c-944e-29e7093a6a92 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:48.804192 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-492a5adb-5b70-4593-b651-c2969f419610 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:48.807919 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fa8b228e-1bdb-431c-944e-29e7093a6a92 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:48.993545 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-492a5adb-5b70-4593-b651-c2969f419610 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:49.006080 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-492a5adb-5b70-4593-b651-c2969f419610 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1275 microversion: 2.1 time: 0.233516 Jul 11 14:40:49.007622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 27/63] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:48 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1275 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:40:49.023234 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9c411d0f-db7c-4080-89a0-9c7dac96f7a2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:49.023879 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-9c411d0f-db7c-4080-89a0-9c7dac96f7a2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:49.048373 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9c411d0f-db7c-4080-89a0-9c7dac96f7a2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:49.049125 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9c411d0f-db7c-4080-89a0-9c7dac96f7a2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:49.050952 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9c411d0f-db7c-4080-89a0-9c7dac96f7a2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:49.054868 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9c411d0f-db7c-4080-89a0-9c7dac96f7a2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:50.003196 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-fa8b228e-1bdb-431c-944e-29e7093a6a92 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:50.015931 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-fa8b228e-1bdb-431c-944e-29e7093a6a92 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1322 microversion: 2.1 time: 1.254251 Jul 11 14:40:50.019945 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 37/64] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:48 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1322 bytes in 1259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:50.032354 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-33622fe3-ec32-420a-be40-fbf88ea4d5b6 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:50.034132 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-33622fe3-ec32-420a-be40-fbf88ea4d5b6 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:50.046723 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-9c411d0f-db7c-4080-89a0-9c7dac96f7a2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:50.060039 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-33622fe3-ec32-420a-be40-fbf88ea4d5b6 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:50.060790 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-33622fe3-ec32-420a-be40-fbf88ea4d5b6 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:50.062611 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9c411d0f-db7c-4080-89a0-9c7dac96f7a2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1241 microversion: 2.1 time: 1.050646 Jul 11 14:40:50.062868 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-33622fe3-ec32-420a-be40-fbf88ea4d5b6 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:50.063146 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-33622fe3-ec32-420a-be40-fbf88ea4d5b6 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:50.064270 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 28/65] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:49 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1241 bytes in 1056 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:50.081693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-48f524a9-5cee-4c6c-99ae-43ca756df835 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:50.083247 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-48f524a9-5cee-4c6c-99ae-43ca756df835 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:50.103903 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-48f524a9-5cee-4c6c-99ae-43ca756df835 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:50.106474 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-48f524a9-5cee-4c6c-99ae-43ca756df835 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:50.107749 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-48f524a9-5cee-4c6c-99ae-43ca756df835 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:50.109075 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-48f524a9-5cee-4c6c-99ae-43ca756df835 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:50.122459 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=100 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:40:50.269318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-33622fe3-ec32-420a-be40-fbf88ea4d5b6 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:50.282770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-33622fe3-ec32-420a-be40-fbf88ea4d5b6 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1302 microversion: 2.1 time: 0.261203 Jul 11 14:40:50.283350 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 38/66] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:50 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1302 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:50.296191 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-0b850edb-aded-4c98-a797-1e5b8e89c46e tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:50.297266 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-0b850edb-aded-4c98-a797-1e5b8e89c46e tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:50.306097 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0b850edb-aded-4c98-a797-1e5b8e89c46e tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:50.306855 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0b850edb-aded-4c98-a797-1e5b8e89c46e tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:50.308368 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0b850edb-aded-4c98-a797-1e5b8e89c46e tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:50.308958 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0b850edb-aded-4c98-a797-1e5b8e89c46e tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:50.358764 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-48f524a9-5cee-4c6c-99ae-43ca756df835 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:50.388628 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-48f524a9-5cee-4c6c-99ae-43ca756df835 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:50.399649 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-48f524a9-5cee-4c6c-99ae-43ca756df835 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:50.404701 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-48f524a9-5cee-4c6c-99ae-43ca756df835 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1333 microversion: 2.20 time: 0.335349 Jul 11 14:40:50.405478 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 29/67] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:40:50 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1333 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:40:50.667570 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0b850edb-aded-4c98-a797-1e5b8e89c46e tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:50.676910 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-0b850edb-aded-4c98-a797-1e5b8e89c46e tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1285 microversion: 2.1 time: 0.391575 Jul 11 14:40:50.677505 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 39/68] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:50 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1285 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:51.038067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c1adcad6-d464-4022-ae98-a6ab8071bac3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:51.038880 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c1adcad6-d464-4022-ae98-a6ab8071bac3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:51.046554 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c1adcad6-d464-4022-ae98-a6ab8071bac3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:51.046914 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c1adcad6-d464-4022-ae98-a6ab8071bac3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:51.047798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c1adcad6-d464-4022-ae98-a6ab8071bac3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:51.048123 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c1adcad6-d464-4022-ae98-a6ab8071bac3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:51.090558 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-64791ba6-7d2f-4814-80c8-20aedd11e2b1 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:51.091426 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-64791ba6-7d2f-4814-80c8-20aedd11e2b1 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:51.103791 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-64791ba6-7d2f-4814-80c8-20aedd11e2b1 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:51.105300 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-64791ba6-7d2f-4814-80c8-20aedd11e2b1 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:51.111932 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-64791ba6-7d2f-4814-80c8-20aedd11e2b1 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:51.113292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-64791ba6-7d2f-4814-80c8-20aedd11e2b1 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:51.206233 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=100 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:40:51.377395 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-64791ba6-7d2f-4814-80c8-20aedd11e2b1 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:51.399009 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-64791ba6-7d2f-4814-80c8-20aedd11e2b1 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1289 microversion: 2.1 time: 0.302293 Jul 11 14:40:51.402284 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 40/69] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:51 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1289 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:40:51.419789 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9f81bed3-5eae-4dee-a9f7-ffd8c9b397c2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:51.421791 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9f81bed3-5eae-4dee-a9f7-ffd8c9b397c2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:51.458243 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9f81bed3-5eae-4dee-a9f7-ffd8c9b397c2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:51.459066 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9f81bed3-5eae-4dee-a9f7-ffd8c9b397c2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:51.460092 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9f81bed3-5eae-4dee-a9f7-ffd8c9b397c2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:51.461127 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9f81bed3-5eae-4dee-a9f7-ffd8c9b397c2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:51.591652 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9f81bed3-5eae-4dee-a9f7-ffd8c9b397c2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:51.601440 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9f81bed3-5eae-4dee-a9f7-ffd8c9b397c2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1300 microversion: 2.1 time: 0.197363 Jul 11 14:40:51.604324 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 41/70] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:51 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1300 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:51.632651 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-0ba4c922-3474-4949-882a-6aa64da248e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:51.634731 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-0ba4c922-3474-4949-882a-6aa64da248e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:51.666895 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0ba4c922-3474-4949-882a-6aa64da248e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:51.672877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0ba4c922-3474-4949-882a-6aa64da248e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:51.677835 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0ba4c922-3474-4949-882a-6aa64da248e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:51.682247 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0ba4c922-3474-4949-882a-6aa64da248e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:51.828424 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0ba4c922-3474-4949-882a-6aa64da248e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:51.847838 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0ba4c922-3474-4949-882a-6aa64da248e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:51.855101 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0ba4c922-3474-4949-882a-6aa64da248e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:51.859324 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-0ba4c922-3474-4949-882a-6aa64da248e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1339 microversion: 2.20 time: 0.247479 Jul 11 14:40:51.860879 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 42/71] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:40:51 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1339 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:40:51.875484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-890231af-d465-4a14-9f59-95ed646146e8 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:51.885502 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-890231af-d465-4a14-9f59-95ed646146e8 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:51.907399 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-890231af-d465-4a14-9f59-95ed646146e8 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:51.910294 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-890231af-d465-4a14-9f59-95ed646146e8 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:51.913784 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-890231af-d465-4a14-9f59-95ed646146e8 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:51.914244 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-890231af-d465-4a14-9f59-95ed646146e8 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:52.100286 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-890231af-d465-4a14-9f59-95ed646146e8 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:52.112958 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c1adcad6-d464-4022-ae98-a6ab8071bac3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:52.113583 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-890231af-d465-4a14-9f59-95ed646146e8 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1273 microversion: 2.1 time: 0.250777 Jul 11 14:40:52.113898 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 43/72] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:51 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1273 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:52.134173 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c1adcad6-d464-4022-ae98-a6ab8071bac3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1322 microversion: 2.1 time: 1.099569 Jul 11 14:40:52.134810 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 30/73] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:51 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1322 bytes in 1101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:52.422413 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-6272454f-3946-4297-8ef8-352890d43974 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:52.423096 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-6272454f-3946-4297-8ef8-352890d43974 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:52.430332 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6272454f-3946-4297-8ef8-352890d43974 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:52.431119 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6272454f-3946-4297-8ef8-352890d43974 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:52.432201 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6272454f-3946-4297-8ef8-352890d43974 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:52.432720 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6272454f-3946-4297-8ef8-352890d43974 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:52.626940 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f00e1ae1-eba1-4442-aa90-b80dbdceaabc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:52.627628 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f00e1ae1-eba1-4442-aa90-b80dbdceaabc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:52.641507 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-6272454f-3946-4297-8ef8-352890d43974 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:52.644184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f00e1ae1-eba1-4442-aa90-b80dbdceaabc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:52.644649 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f00e1ae1-eba1-4442-aa90-b80dbdceaabc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:52.659983 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-6272454f-3946-4297-8ef8-352890d43974 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1297 microversion: 2.1 time: 0.246640 Jul 11 14:40:52.661270 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 44/74] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:52 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1297 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:52.663876 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f00e1ae1-eba1-4442-aa90-b80dbdceaabc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:52.664632 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f00e1ae1-eba1-4442-aa90-b80dbdceaabc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:52.882893 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-863a61fb-9d57-45ec-96ff-7e541f37c371 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:52.885184 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-863a61fb-9d57-45ec-96ff-7e541f37c371 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:52.895908 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-863a61fb-9d57-45ec-96ff-7e541f37c371 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:52.896712 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-863a61fb-9d57-45ec-96ff-7e541f37c371 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:52.898083 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-863a61fb-9d57-45ec-96ff-7e541f37c371 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:52.898727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-863a61fb-9d57-45ec-96ff-7e541f37c371 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:53.039012 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-863a61fb-9d57-45ec-96ff-7e541f37c371 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:53.047373 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-863a61fb-9d57-45ec-96ff-7e541f37c371 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:53.049224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-863a61fb-9d57-45ec-96ff-7e541f37c371 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:53.050388 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-863a61fb-9d57-45ec-96ff-7e541f37c371 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1339 microversion: 2.20 time: 0.175831 Jul 11 14:40:53.054593 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 45/75] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:40:52 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1339 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:40:53.137381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-232b34d3-0eff-4786-91b4-315cfaa9c7cd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:53.139683 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-232b34d3-0eff-4786-91b4-315cfaa9c7cd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:53.150096 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-232b34d3-0eff-4786-91b4-315cfaa9c7cd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:53.150966 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-232b34d3-0eff-4786-91b4-315cfaa9c7cd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:53.152541 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-232b34d3-0eff-4786-91b4-315cfaa9c7cd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:53.153367 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-232b34d3-0eff-4786-91b4-315cfaa9c7cd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:53.480927 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f00e1ae1-eba1-4442-aa90-b80dbdceaabc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:53.488625 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f00e1ae1-eba1-4442-aa90-b80dbdceaabc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1368 microversion: 2.1 time: 0.869804 Jul 11 14:40:53.489197 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 31/76] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:52 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1368 bytes in 871 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:53.494144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-0ecd1380-ecff-48f5-b27a-9a5f6aef14c7 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:53.495713 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-0ecd1380-ecff-48f5-b27a-9a5f6aef14c7 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:53.514800 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0ecd1380-ecff-48f5-b27a-9a5f6aef14c7 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:53.515630 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0ecd1380-ecff-48f5-b27a-9a5f6aef14c7 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:53.516647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0ecd1380-ecff-48f5-b27a-9a5f6aef14c7 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:53.517288 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0ecd1380-ecff-48f5-b27a-9a5f6aef14c7 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:53.729917 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-232b34d3-0eff-4786-91b4-315cfaa9c7cd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:53.742642 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-232b34d3-0eff-4786-91b4-315cfaa9c7cd tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1315 microversion: 2.1 time: 0.610934 Jul 11 14:40:53.743712 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 46/77] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:53 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1315 bytes in 617 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:53.748616 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-61aab34c-ed2b-4005-940c-6f5459c1344d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:53.749237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-61aab34c-ed2b-4005-940c-6f5459c1344d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:53.759640 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-61aab34c-ed2b-4005-940c-6f5459c1344d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:53.760292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-61aab34c-ed2b-4005-940c-6f5459c1344d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:53.762026 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-61aab34c-ed2b-4005-940c-6f5459c1344d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:53.762816 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-61aab34c-ed2b-4005-940c-6f5459c1344d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:53.891740 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-61aab34c-ed2b-4005-940c-6f5459c1344d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:53.901915 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-61aab34c-ed2b-4005-940c-6f5459c1344d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1295 microversion: 2.1 time: 0.156666 Jul 11 14:40:53.902893 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 47/78] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:53 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1295 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:54.003617 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-0ecd1380-ecff-48f5-b27a-9a5f6aef14c7 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:54.013162 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-0ecd1380-ecff-48f5-b27a-9a5f6aef14c7 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1322 microversion: 2.1 time: 0.522411 Jul 11 14:40:54.013766 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 32/79] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:53 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1322 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:54.068787 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d1b7581a-ec35-4d5d-80ab-685efdc3de14 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:54.069822 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d1b7581a-ec35-4d5d-80ab-685efdc3de14 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:54.076042 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d1b7581a-ec35-4d5d-80ab-685efdc3de14 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:54.076837 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d1b7581a-ec35-4d5d-80ab-685efdc3de14 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:54.078963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d1b7581a-ec35-4d5d-80ab-685efdc3de14 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:54.079747 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d1b7581a-ec35-4d5d-80ab-685efdc3de14 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:54.228481 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d1b7581a-ec35-4d5d-80ab-685efdc3de14 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:54.238803 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=4 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:40:54.251038 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d1b7581a-ec35-4d5d-80ab-685efdc3de14 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:54.253488 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d1b7581a-ec35-4d5d-80ab-685efdc3de14 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:54.260334 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-d1b7581a-ec35-4d5d-80ab-685efdc3de14 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1339 microversion: 2.20 time: 0.193906 Jul 11 14:40:54.263287 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 48/80] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:40:54 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1339 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:40:54.335822 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-3a7cc119-1a60-4d64-9e16-08d28b5222e0 req-d7bc09b6-da53-4ca9-a749-6144fb8425e3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a3e173ea-1d52-48e5-a5d1-89195af310c7", "tag": "5f8fa020-0a9e-499b-a83e-8e2d443ee007"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:40:54.359491 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-3a7cc119-1a60-4d64-9e16-08d28b5222e0 req-d7bc09b6-da53-4ca9-a749-6144fb8425e3 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:54.369265 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-3a7cc119-1a60-4d64-9e16-08d28b5222e0 req-d7bc09b6-da53-4ca9-a749-6144fb8425e3 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:54.409285 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-3a7cc119-1a60-4d64-9e16-08d28b5222e0 req-d7bc09b6-da53-4ca9-a749-6144fb8425e3 service nova] Creating event network-changed:5f8fa020-0a9e-499b-a83e-8e2d443ee007 for instance a3e173ea-1d52-48e5-a5d1-89195af310c7 on hv-cinder-maste Jul 11 14:40:54.437048 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-3a7cc119-1a60-4d64-9e16-08d28b5222e0 req-d7bc09b6-da53-4ca9-a749-6144fb8425e3 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.099912 Jul 11 14:40:54.442283 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 33/81] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:40:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:40:54.508926 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-31462247-7b5d-4bdb-bc13-0c57f3df30ea tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:54.510066 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-31462247-7b5d-4bdb-bc13-0c57f3df30ea tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:54.519397 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-31462247-7b5d-4bdb-bc13-0c57f3df30ea tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:54.520225 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-31462247-7b5d-4bdb-bc13-0c57f3df30ea tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:54.521222 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-31462247-7b5d-4bdb-bc13-0c57f3df30ea tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:54.522137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-31462247-7b5d-4bdb-bc13-0c57f3df30ea tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:54.760396 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-1fc07747-ba0b-4410-8df0-aeba96d8a561 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:54.761135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-1fc07747-ba0b-4410-8df0-aeba96d8a561 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:54.768499 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1fc07747-ba0b-4410-8df0-aeba96d8a561 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:54.768950 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1fc07747-ba0b-4410-8df0-aeba96d8a561 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:54.770079 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1fc07747-ba0b-4410-8df0-aeba96d8a561 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:54.773110 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1fc07747-ba0b-4410-8df0-aeba96d8a561 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:54.932198 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-31462247-7b5d-4bdb-bc13-0c57f3df30ea tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:54.941893 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-31462247-7b5d-4bdb-bc13-0c57f3df30ea tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1368 microversion: 2.1 time: 0.440206 Jul 11 14:40:54.954821 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 49/82] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:54 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1368 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:54.960459 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-901bc6f1-f134-467d-a2c0-af65516cf429 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:54.961093 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-901bc6f1-f134-467d-a2c0-af65516cf429 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:54.989855 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-901bc6f1-f134-467d-a2c0-af65516cf429 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:54.991958 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-901bc6f1-f134-467d-a2c0-af65516cf429 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:54.992863 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-901bc6f1-f134-467d-a2c0-af65516cf429 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:54.993387 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-901bc6f1-f134-467d-a2c0-af65516cf429 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:55.045165 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-1fc07747-ba0b-4410-8df0-aeba96d8a561 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:55.066715 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-1fc07747-ba0b-4410-8df0-aeba96d8a561 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1315 microversion: 2.1 time: 0.310473 Jul 11 14:40:55.077488 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 34/83] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:54 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1315 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:40:55.090775 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-4b1a3c51-f464-4fed-b580-4d30b116a23b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:55.093669 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-4b1a3c51-f464-4fed-b580-4d30b116a23b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:55.101382 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4b1a3c51-f464-4fed-b580-4d30b116a23b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:55.102177 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4b1a3c51-f464-4fed-b580-4d30b116a23b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:55.103144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4b1a3c51-f464-4fed-b580-4d30b116a23b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:55.103736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4b1a3c51-f464-4fed-b580-4d30b116a23b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:55.125947 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-901bc6f1-f134-467d-a2c0-af65516cf429 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:55.136352 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-901bc6f1-f134-467d-a2c0-af65516cf429 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1295 microversion: 2.1 time: 0.179621 Jul 11 14:40:55.139794 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 50/84] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:54 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1295 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:55.160501 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-b554ec13-cf03-480f-b182-eaae25ab97ae req-057b8892-044b-4df1-baeb-21144e298b9f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bd855cba-6708-47de-826b-00c5a74b23f2", "tag": "8b901616-9eed-4045-b039-9047df0bd8e7"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:40:55.195518 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-b554ec13-cf03-480f-b182-eaae25ab97ae req-057b8892-044b-4df1-baeb-21144e298b9f service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:55.196031 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-b554ec13-cf03-480f-b182-eaae25ab97ae req-057b8892-044b-4df1-baeb-21144e298b9f service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:55.226211 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-b554ec13-cf03-480f-b182-eaae25ab97ae req-057b8892-044b-4df1-baeb-21144e298b9f service nova] Creating event network-changed:8b901616-9eed-4045-b039-9047df0bd8e7 for instance bd855cba-6708-47de-826b-00c5a74b23f2 on hv-cinder-maste Jul 11 14:40:55.238086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-b554ec13-cf03-480f-b182-eaae25ab97ae req-057b8892-044b-4df1-baeb-21144e298b9f service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.087095 Jul 11 14:40:55.240078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 51/85] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:40:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:40:55.282678 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-7be3d00d-bde0-4019-bbe5-a70ffac87adc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:55.283703 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-7be3d00d-bde0-4019-bbe5-a70ffac87adc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:55.290911 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7be3d00d-bde0-4019-bbe5-a70ffac87adc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:55.291601 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7be3d00d-bde0-4019-bbe5-a70ffac87adc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:55.295457 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7be3d00d-bde0-4019-bbe5-a70ffac87adc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:55.296121 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7be3d00d-bde0-4019-bbe5-a70ffac87adc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:55.327110 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=12 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:40:55.350205 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=32 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:40:55.365020 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4b1a3c51-f464-4fed-b580-4d30b116a23b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:55.382256 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-4b1a3c51-f464-4fed-b580-4d30b116a23b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1322 microversion: 2.1 time: 0.303364 Jul 11 14:40:55.385288 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 35/86] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:55 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1322 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:55.522923 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=30 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:40:55.724602 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7be3d00d-bde0-4019-bbe5-a70ffac87adc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:55.734772 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7be3d00d-bde0-4019-bbe5-a70ffac87adc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:55.736318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7be3d00d-bde0-4019-bbe5-a70ffac87adc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:55.737932 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-7be3d00d-bde0-4019-bbe5-a70ffac87adc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1407 microversion: 2.20 time: 0.460345 Jul 11 14:40:55.738522 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 52/87] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:40:55 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1407 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:40:55.961135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d2bfdcc2-ac43-44a0-ac41-51c45408f934 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:55.961836 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d2bfdcc2-ac43-44a0-ac41-51c45408f934 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:55.971482 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d2bfdcc2-ac43-44a0-ac41-51c45408f934 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:55.972262 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d2bfdcc2-ac43-44a0-ac41-51c45408f934 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:55.973315 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d2bfdcc2-ac43-44a0-ac41-51c45408f934 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:55.973999 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d2bfdcc2-ac43-44a0-ac41-51c45408f934 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:56.085896 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c83d5c7a-fae4-4781-b1f3-4f811310dd8d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:56.086528 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c83d5c7a-fae4-4781-b1f3-4f811310dd8d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:56.093530 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c83d5c7a-fae4-4781-b1f3-4f811310dd8d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:56.094073 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c83d5c7a-fae4-4781-b1f3-4f811310dd8d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:56.094951 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c83d5c7a-fae4-4781-b1f3-4f811310dd8d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:56.095758 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c83d5c7a-fae4-4781-b1f3-4f811310dd8d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:56.135106 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d2bfdcc2-ac43-44a0-ac41-51c45408f934 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:56.146624 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d2bfdcc2-ac43-44a0-ac41-51c45408f934 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1368 microversion: 2.1 time: 0.188584 Jul 11 14:40:56.146828 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 36/88] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:55 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1368 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:56.161355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-946eccd1-91e9-4351-ba04-c70e0039b892 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:56.162315 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-946eccd1-91e9-4351-ba04-c70e0039b892 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:56.170418 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-946eccd1-91e9-4351-ba04-c70e0039b892 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:56.171879 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-946eccd1-91e9-4351-ba04-c70e0039b892 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:56.173226 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-946eccd1-91e9-4351-ba04-c70e0039b892 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:56.173879 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-946eccd1-91e9-4351-ba04-c70e0039b892 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:56.436263 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-946eccd1-91e9-4351-ba04-c70e0039b892 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:56.444752 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-946eccd1-91e9-4351-ba04-c70e0039b892 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1295 microversion: 2.1 time: 0.289103 Jul 11 14:40:56.454835 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 37/89] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:56 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1295 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:56.461484 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2a114276-0288-4973-ac0e-07f87bd073a8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:56.462313 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2a114276-0288-4973-ac0e-07f87bd073a8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:56.477887 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2a114276-0288-4973-ac0e-07f87bd073a8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:56.478659 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2a114276-0288-4973-ac0e-07f87bd073a8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:56.479681 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2a114276-0288-4973-ac0e-07f87bd073a8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:56.480325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2a114276-0288-4973-ac0e-07f87bd073a8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:56.716757 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c83d5c7a-fae4-4781-b1f3-4f811310dd8d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:56.725618 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c83d5c7a-fae4-4781-b1f3-4f811310dd8d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1315 microversion: 2.1 time: 0.643851 Jul 11 14:40:56.736103 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 53/90] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:56 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1315 bytes in 655 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:56.756923 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-31bc8428-d21d-4b21-8725-cf6b609f4b18 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:56.757758 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-31bc8428-d21d-4b21-8725-cf6b609f4b18 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:56.774362 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-31bc8428-d21d-4b21-8725-cf6b609f4b18 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:56.774846 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-31bc8428-d21d-4b21-8725-cf6b609f4b18 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:56.775624 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-31bc8428-d21d-4b21-8725-cf6b609f4b18 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:56.776281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-31bc8428-d21d-4b21-8725-cf6b609f4b18 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:57.101172 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2a114276-0288-4973-ac0e-07f87bd073a8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:57.111844 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2a114276-0288-4973-ac0e-07f87bd073a8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1322 microversion: 2.1 time: 0.655442 Jul 11 14:40:57.113194 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 38/91] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:56 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1322 bytes in 657 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:57.164734 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-cd799308-eda5-4c21-bfad-b255fc928552 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:57.166864 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-cd799308-eda5-4c21-bfad-b255fc928552 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:57.188212 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cd799308-eda5-4c21-bfad-b255fc928552 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:57.188873 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cd799308-eda5-4c21-bfad-b255fc928552 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:57.189657 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cd799308-eda5-4c21-bfad-b255fc928552 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:57.190145 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cd799308-eda5-4c21-bfad-b255fc928552 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:57.375133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-31bc8428-d21d-4b21-8725-cf6b609f4b18 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:57.389837 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-31bc8428-d21d-4b21-8725-cf6b609f4b18 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:57.395260 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-31bc8428-d21d-4b21-8725-cf6b609f4b18 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:57.400062 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-31bc8428-d21d-4b21-8725-cf6b609f4b18 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1407 microversion: 2.20 time: 0.647361 Jul 11 14:40:57.403239 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 54/92] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:40:56 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1407 bytes in 651 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:40:57.471954 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c7610d89-750f-47a8-9b3a-ee417ee44af8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:57.476079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c7610d89-750f-47a8-9b3a-ee417ee44af8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:57.483227 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c7610d89-750f-47a8-9b3a-ee417ee44af8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:57.483799 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c7610d89-750f-47a8-9b3a-ee417ee44af8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:57.484691 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c7610d89-750f-47a8-9b3a-ee417ee44af8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:57.485233 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c7610d89-750f-47a8-9b3a-ee417ee44af8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:57.580944 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c7610d89-750f-47a8-9b3a-ee417ee44af8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:57.581364 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-cd799308-eda5-4c21-bfad-b255fc928552 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:57.603649 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-cd799308-eda5-4c21-bfad-b255fc928552 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1368 microversion: 2.1 time: 0.439765 Jul 11 14:40:57.605551 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 39/93] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:57 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1368 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:40:57.607749 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c7610d89-750f-47a8-9b3a-ee417ee44af8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1295 microversion: 2.1 time: 0.140414 Jul 11 14:40:57.610317 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 55/94] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:57 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1295 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:57.749673 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-dd6c1a5e-91a0-44d4-9fae-34e0f78b95aa tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:57.750294 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-dd6c1a5e-91a0-44d4-9fae-34e0f78b95aa tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:57.761277 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dd6c1a5e-91a0-44d4-9fae-34e0f78b95aa tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:57.761728 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dd6c1a5e-91a0-44d4-9fae-34e0f78b95aa tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:57.762540 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dd6c1a5e-91a0-44d4-9fae-34e0f78b95aa tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:57.769602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dd6c1a5e-91a0-44d4-9fae-34e0f78b95aa tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:58.098067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-dd6c1a5e-91a0-44d4-9fae-34e0f78b95aa tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:58.118503 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-dd6c1a5e-91a0-44d4-9fae-34e0f78b95aa tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1315 microversion: 2.1 time: 0.376163 Jul 11 14:40:58.120239 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 40/95] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:57 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1315 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:58.129891 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e5ca039c-783f-4505-a25f-5c819d248ea8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:58.131123 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e5ca039c-783f-4505-a25f-5c819d248ea8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:58.138217 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e5ca039c-783f-4505-a25f-5c819d248ea8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:58.143547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e5ca039c-783f-4505-a25f-5c819d248ea8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:58.144478 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e5ca039c-783f-4505-a25f-5c819d248ea8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:58.144932 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e5ca039c-783f-4505-a25f-5c819d248ea8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:58.422386 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-678329a9-b1bb-4a9c-9e9a-73a22baa0722 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:58.423865 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-678329a9-b1bb-4a9c-9e9a-73a22baa0722 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:58.432836 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-678329a9-b1bb-4a9c-9e9a-73a22baa0722 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:58.435886 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-678329a9-b1bb-4a9c-9e9a-73a22baa0722 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:58.439364 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-678329a9-b1bb-4a9c-9e9a-73a22baa0722 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:58.440659 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-678329a9-b1bb-4a9c-9e9a-73a22baa0722 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:58.581894 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e5ca039c-783f-4505-a25f-5c819d248ea8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:58.594948 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e5ca039c-783f-4505-a25f-5c819d248ea8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1322 microversion: 2.1 time: 0.466647 Jul 11 14:40:58.604086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 56/96] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:58 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1322 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:58.627148 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8e9d6d42-8ddb-4822-9c6e-341f1608fe69 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:58.628203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8e9d6d42-8ddb-4822-9c6e-341f1608fe69 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:58.635627 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8e9d6d42-8ddb-4822-9c6e-341f1608fe69 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:58.636344 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8e9d6d42-8ddb-4822-9c6e-341f1608fe69 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:58.637378 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8e9d6d42-8ddb-4822-9c6e-341f1608fe69 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:58.637973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8e9d6d42-8ddb-4822-9c6e-341f1608fe69 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:58.968232 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-678329a9-b1bb-4a9c-9e9a-73a22baa0722 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:58.982127 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-678329a9-b1bb-4a9c-9e9a-73a22baa0722 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:58.986661 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-678329a9-b1bb-4a9c-9e9a-73a22baa0722 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:58.990003 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-678329a9-b1bb-4a9c-9e9a-73a22baa0722 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1407 microversion: 2.20 time: 0.572008 Jul 11 14:40:58.992184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 41/97] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:40:58 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1407 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:40:58.997036 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2218f6fa-c245-4269-81f9-f97ea4da0868 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:59.004705 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2218f6fa-c245-4269-81f9-f97ea4da0868 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:59.022134 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2218f6fa-c245-4269-81f9-f97ea4da0868 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:59.028667 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2218f6fa-c245-4269-81f9-f97ea4da0868 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:59.029933 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2218f6fa-c245-4269-81f9-f97ea4da0868 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:59.030561 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2218f6fa-c245-4269-81f9-f97ea4da0868 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:59.183440 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8e9d6d42-8ddb-4822-9c6e-341f1608fe69 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:59.191791 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=100 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:40:59.199599 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8e9d6d42-8ddb-4822-9c6e-341f1608fe69 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1361 microversion: 2.1 time: 0.575040 Jul 11 14:40:59.203060 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 57/98] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:58 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1361 bytes in 580 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:40:59.212366 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-24a0b11f-28dc-48aa-975f-1c265cd08f11 req-9aca4b79-e07d-454b-b54a-cd4502aaef5f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "13da005a-e085-43aa-8a18-1fe85e4e95af", "tag": "7b725b13-befd-4a14-98cc-9d0e599f5e17"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:40:59.236271 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-24a0b11f-28dc-48aa-975f-1c265cd08f11 req-9aca4b79-e07d-454b-b54a-cd4502aaef5f service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:59.237344 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-24a0b11f-28dc-48aa-975f-1c265cd08f11 req-9aca4b79-e07d-454b-b54a-cd4502aaef5f service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:59.261370 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-24a0b11f-28dc-48aa-975f-1c265cd08f11 req-9aca4b79-e07d-454b-b54a-cd4502aaef5f service nova] Creating event network-changed:7b725b13-befd-4a14-98cc-9d0e599f5e17 for instance 13da005a-e085-43aa-8a18-1fe85e4e95af on hv-cinder-maste Jul 11 14:40:59.274557 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-24a0b11f-28dc-48aa-975f-1c265cd08f11 req-9aca4b79-e07d-454b-b54a-cd4502aaef5f service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.069018 Jul 11 14:40:59.284750 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 58/99] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:40:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:40:59.298184 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-32136398-8418-46b4-889c-8153ca5464ca tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:59.299990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-32136398-8418-46b4-889c-8153ca5464ca tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:59.315352 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-32136398-8418-46b4-889c-8153ca5464ca tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:59.316067 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-32136398-8418-46b4-889c-8153ca5464ca tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:59.317163 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-32136398-8418-46b4-889c-8153ca5464ca tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:59.324540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-32136398-8418-46b4-889c-8153ca5464ca tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:59.626131 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-32136398-8418-46b4-889c-8153ca5464ca tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:59.631743 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2218f6fa-c245-4269-81f9-f97ea4da0868 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:40:59.640614 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-32136398-8418-46b4-889c-8153ca5464ca tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1315 microversion: 2.1 time: 0.354267 Jul 11 14:40:59.641146 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2218f6fa-c245-4269-81f9-f97ea4da0868 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1368 microversion: 2.1 time: 0.647754 Jul 11 14:40:59.651118 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 42/100] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:58 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1368 bytes in 658 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:59.652258 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 59/101] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:59 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1315 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:40:59.657074 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-51d98879-95f0-42e9-b943-2e66306b9f68 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:40:59.657899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-51d98879-95f0-42e9-b943-2e66306b9f68 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:40:59.684981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-51d98879-95f0-42e9-b943-2e66306b9f68 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:59.687881 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-51d98879-95f0-42e9-b943-2e66306b9f68 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:59.689867 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-51d98879-95f0-42e9-b943-2e66306b9f68 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:59.690899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-51d98879-95f0-42e9-b943-2e66306b9f68 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:59.780294 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-827bd354-0fbb-4547-8e47-3418ebd99d3c req-37a07b0a-75f8-44f0-9dc8-c6e618c1349b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cc0591e3-09b9-41cd-b222-f730282f8572", "name": "network-vif-plugged", "status": "completed", "tag": "9af04c14-ce8d-46f1-8097-0f76fe117ff9"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:40:59.807254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-827bd354-0fbb-4547-8e47-3418ebd99d3c req-37a07b0a-75f8-44f0-9dc8-c6e618c1349b service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:40:59.807829 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-827bd354-0fbb-4547-8e47-3418ebd99d3c req-37a07b0a-75f8-44f0-9dc8-c6e618c1349b service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:40:59.823418 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-827bd354-0fbb-4547-8e47-3418ebd99d3c req-37a07b0a-75f8-44f0-9dc8-c6e618c1349b service nova] Creating event network-vif-plugged:9af04c14-ce8d-46f1-8097-0f76fe117ff9 for instance cc0591e3-09b9-41cd-b222-f730282f8572 on hv-cinder-maste Jul 11 14:40:59.829366 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-827bd354-0fbb-4547-8e47-3418ebd99d3c req-37a07b0a-75f8-44f0-9dc8-c6e618c1349b service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.176306 Jul 11 14:40:59.832121 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 43/102] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:40:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:41:00.015158 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-accd1527-f649-4a45-9b41-2e217602ead7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:00.016270 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-accd1527-f649-4a45-9b41-2e217602ead7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:00.052305 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-accd1527-f649-4a45-9b41-2e217602ead7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:00.057972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-accd1527-f649-4a45-9b41-2e217602ead7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:00.060204 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-accd1527-f649-4a45-9b41-2e217602ead7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:00.061345 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-accd1527-f649-4a45-9b41-2e217602ead7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:00.080936 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-51d98879-95f0-42e9-b943-2e66306b9f68 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:00.089466 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-51d98879-95f0-42e9-b943-2e66306b9f68 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1322 microversion: 2.1 time: 0.435502 Jul 11 14:41:00.098392 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 60/103] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:40:59 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1322 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:00.218277 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-ecaf50e0-13a9-49d2-8b94-52f9ec43bb68 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:00.218959 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-ecaf50e0-13a9-49d2-8b94-52f9ec43bb68 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:00.232584 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ecaf50e0-13a9-49d2-8b94-52f9ec43bb68 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:00.241596 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ecaf50e0-13a9-49d2-8b94-52f9ec43bb68 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:00.246155 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ecaf50e0-13a9-49d2-8b94-52f9ec43bb68 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:00.248172 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ecaf50e0-13a9-49d2-8b94-52f9ec43bb68 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:00.420363 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-accd1527-f649-4a45-9b41-2e217602ead7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:00.442968 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-accd1527-f649-4a45-9b41-2e217602ead7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:00.447025 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-accd1527-f649-4a45-9b41-2e217602ead7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:00.461732 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-accd1527-f649-4a45-9b41-2e217602ead7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1407 microversion: 2.20 time: 0.452433 Jul 11 14:41:00.474517 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 44/104] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:41:00 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1407 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:41:00.484633 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-0c54079b-aa01-4ce1-b8dc-1eebdfb84ebd req-47354c34-b6b0-412f-9f43-65437207e974 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "62a90c58-0027-47e8-8909-44512060e08d", "tag": "edfcc9e3-9cd5-4b35-adc9-68a5f06aeb87"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:41:00.495436 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-0c54079b-aa01-4ce1-b8dc-1eebdfb84ebd req-47354c34-b6b0-412f-9f43-65437207e974 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:00.496144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-0c54079b-aa01-4ce1-b8dc-1eebdfb84ebd req-47354c34-b6b0-412f-9f43-65437207e974 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:00.517327 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-0c54079b-aa01-4ce1-b8dc-1eebdfb84ebd req-47354c34-b6b0-412f-9f43-65437207e974 service nova] Creating event network-changed:edfcc9e3-9cd5-4b35-adc9-68a5f06aeb87 for instance 62a90c58-0027-47e8-8909-44512060e08d on hv-cinder-maste Jul 11 14:41:00.526106 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-0c54079b-aa01-4ce1-b8dc-1eebdfb84ebd req-47354c34-b6b0-412f-9f43-65437207e974 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049106 Jul 11 14:41:00.533726 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 45/105] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:41:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:41:00.614186 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-ecaf50e0-13a9-49d2-8b94-52f9ec43bb68 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:00.622905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-ecaf50e0-13a9-49d2-8b94-52f9ec43bb68 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1361 microversion: 2.1 time: 0.409515 Jul 11 14:41:00.623496 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 61/106] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:00 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1361 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:41:00.660884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7439ed67-61cf-4a03-8a73-574ccbeeee5d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:00.661931 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-7439ed67-61cf-4a03-8a73-574ccbeeee5d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:00.680067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7439ed67-61cf-4a03-8a73-574ccbeeee5d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:00.681108 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7439ed67-61cf-4a03-8a73-574ccbeeee5d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:00.683330 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2543499e-368b-44c6-8410-2c184098ffb4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:00.683727 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7439ed67-61cf-4a03-8a73-574ccbeeee5d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:00.684187 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7439ed67-61cf-4a03-8a73-574ccbeeee5d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:00.685968 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2543499e-368b-44c6-8410-2c184098ffb4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:00.700455 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2543499e-368b-44c6-8410-2c184098ffb4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:00.700916 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2543499e-368b-44c6-8410-2c184098ffb4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:00.701824 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2543499e-368b-44c6-8410-2c184098ffb4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:00.702453 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2543499e-368b-44c6-8410-2c184098ffb4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:00.919344 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7439ed67-61cf-4a03-8a73-574ccbeeee5d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:00.929923 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-7439ed67-61cf-4a03-8a73-574ccbeeee5d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1368 microversion: 2.1 time: 0.274176 Jul 11 14:41:00.930826 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 46/107] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:00 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1368 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:00.955932 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-2543499e-368b-44c6-8410-2c184098ffb4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:00.965327 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2543499e-368b-44c6-8410-2c184098ffb4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1315 microversion: 2.1 time: 0.294128 Jul 11 14:41:00.965909 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 62/108] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:00 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1315 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:01.118073 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e26c93ed-1a68-40d6-ad16-2cca1dc21d40 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:01.119290 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-e26c93ed-1a68-40d6-ad16-2cca1dc21d40 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:01.127457 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e26c93ed-1a68-40d6-ad16-2cca1dc21d40 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:01.128020 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e26c93ed-1a68-40d6-ad16-2cca1dc21d40 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:01.128901 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e26c93ed-1a68-40d6-ad16-2cca1dc21d40 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:01.129576 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e26c93ed-1a68-40d6-ad16-2cca1dc21d40 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:01.348262 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e26c93ed-1a68-40d6-ad16-2cca1dc21d40 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:01.356378 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=100 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:41:01.361499 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e26c93ed-1a68-40d6-ad16-2cca1dc21d40 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1322 microversion: 2.1 time: 0.248055 Jul 11 14:41:01.371519 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 47/109] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:01 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1322 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:01.482758 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f7cea853-52fc-4d98-b4c7-30c451ee1203 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:01.483801 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f7cea853-52fc-4d98-b4c7-30c451ee1203 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:01.491047 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f7cea853-52fc-4d98-b4c7-30c451ee1203 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:01.491565 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f7cea853-52fc-4d98-b4c7-30c451ee1203 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:01.492526 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f7cea853-52fc-4d98-b4c7-30c451ee1203 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:01.493352 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f7cea853-52fc-4d98-b4c7-30c451ee1203 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:01.644351 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-10d2fadd-3c88-4037-a4f7-8a586a645950 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:01.645090 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-10d2fadd-3c88-4037-a4f7-8a586a645950 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:01.646017 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f7cea853-52fc-4d98-b4c7-30c451ee1203 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:01.654933 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f7cea853-52fc-4d98-b4c7-30c451ee1203 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:01.655271 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-10d2fadd-3c88-4037-a4f7-8a586a645950 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:01.656560 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-10d2fadd-3c88-4037-a4f7-8a586a645950 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:01.657316 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f7cea853-52fc-4d98-b4c7-30c451ee1203 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:01.657437 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-10d2fadd-3c88-4037-a4f7-8a586a645950 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:01.657816 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-10d2fadd-3c88-4037-a4f7-8a586a645950 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:01.659162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f7cea853-52fc-4d98-b4c7-30c451ee1203 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1407 microversion: 2.20 time: 0.180187 Jul 11 14:41:01.660315 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 63/110] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:41:01 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1407 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:41:01.830753 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-10d2fadd-3c88-4037-a4f7-8a586a645950 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:01.840362 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-10d2fadd-3c88-4037-a4f7-8a586a645950 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1361 microversion: 2.1 time: 0.203423 Jul 11 14:41:01.841285 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 48/111] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:01 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1361 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:01.853056 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-d727ee92-8041-4d38-acc9-bd580a2453ba req-1459e7bd-b1d7-4edf-8994-b3189dc68527 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a0498735-9e0a-4f14-990d-a6d5076e742a", "name": "network-vif-plugged", "status": "completed", "tag": "27aef129-6bd8-4686-9e6d-915b57154c5a"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:41:01.862116 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-d727ee92-8041-4d38-acc9-bd580a2453ba req-1459e7bd-b1d7-4edf-8994-b3189dc68527 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:01.862917 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-d727ee92-8041-4d38-acc9-bd580a2453ba req-1459e7bd-b1d7-4edf-8994-b3189dc68527 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:01.876096 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-d727ee92-8041-4d38-acc9-bd580a2453ba req-1459e7bd-b1d7-4edf-8994-b3189dc68527 service nova] Creating event network-vif-plugged:27aef129-6bd8-4686-9e6d-915b57154c5a for instance a0498735-9e0a-4f14-990d-a6d5076e742a on hv-cinder-maste Jul 11 14:41:01.881419 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-d727ee92-8041-4d38-acc9-bd580a2453ba req-1459e7bd-b1d7-4edf-8994-b3189dc68527 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032182 Jul 11 14:41:01.882513 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 64/112] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:41:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:41:01.950081 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-eef4c4ee-a7cd-453d-aff9-fa02feb91329 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:01.951107 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-eef4c4ee-a7cd-453d-aff9-fa02feb91329 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:01.963028 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eef4c4ee-a7cd-453d-aff9-fa02feb91329 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:01.963912 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eef4c4ee-a7cd-453d-aff9-fa02feb91329 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:01.965144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eef4c4ee-a7cd-453d-aff9-fa02feb91329 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:01.966086 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eef4c4ee-a7cd-453d-aff9-fa02feb91329 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:01.983785 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2484fb2c-032a-474f-aebc-712e4f998209 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:01.984558 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2484fb2c-032a-474f-aebc-712e4f998209 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:01.991135 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2484fb2c-032a-474f-aebc-712e4f998209 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:01.991595 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2484fb2c-032a-474f-aebc-712e4f998209 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:01.992364 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2484fb2c-032a-474f-aebc-712e4f998209 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:01.993033 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2484fb2c-032a-474f-aebc-712e4f998209 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:02.201203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-2484fb2c-032a-474f-aebc-712e4f998209 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:02.201858 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-eef4c4ee-a7cd-453d-aff9-fa02feb91329 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:02.211124 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2484fb2c-032a-474f-aebc-712e4f998209 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1315 microversion: 2.1 time: 0.232203 Jul 11 14:41:02.212593 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-eef4c4ee-a7cd-453d-aff9-fa02feb91329 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1368 microversion: 2.1 time: 0.267669 Jul 11 14:41:02.212808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 65/113] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:01 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1315 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:41:02.215918 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 49/114] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:01 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1368 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:02.379650 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9bb389ab-43c7-4ccb-9d8b-d346a874dd07 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:02.380541 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9bb389ab-43c7-4ccb-9d8b-d346a874dd07 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:02.387042 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9bb389ab-43c7-4ccb-9d8b-d346a874dd07 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:02.388178 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9bb389ab-43c7-4ccb-9d8b-d346a874dd07 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:02.388987 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9bb389ab-43c7-4ccb-9d8b-d346a874dd07 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:02.389429 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9bb389ab-43c7-4ccb-9d8b-d346a874dd07 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:02.556532 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9bb389ab-43c7-4ccb-9d8b-d346a874dd07 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:02.574906 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9bb389ab-43c7-4ccb-9d8b-d346a874dd07 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1322 microversion: 2.1 time: 0.199603 Jul 11 14:41:02.575712 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 66/115] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:02 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1322 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:02.677393 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-6d2bc690-a6a0-4343-ac9d-f6b5c881e043 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:02.677942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-6d2bc690-a6a0-4343-ac9d-f6b5c881e043 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:02.684223 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6d2bc690-a6a0-4343-ac9d-f6b5c881e043 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:02.684789 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6d2bc690-a6a0-4343-ac9d-f6b5c881e043 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:02.685645 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6d2bc690-a6a0-4343-ac9d-f6b5c881e043 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:02.686196 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6d2bc690-a6a0-4343-ac9d-f6b5c881e043 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:02.853193 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6d2bc690-a6a0-4343-ac9d-f6b5c881e043 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:02.864681 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c620abcc-0e16-489d-ad09-17c12ff8a48e tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:02.865594 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c620abcc-0e16-489d-ad09-17c12ff8a48e tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:02.877378 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c620abcc-0e16-489d-ad09-17c12ff8a48e tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:02.877751 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6d2bc690-a6a0-4343-ac9d-f6b5c881e043 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:02.879584 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6d2bc690-a6a0-4343-ac9d-f6b5c881e043 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:02.880258 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c620abcc-0e16-489d-ad09-17c12ff8a48e tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:02.881032 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c620abcc-0e16-489d-ad09-17c12ff8a48e tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:02.881814 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c620abcc-0e16-489d-ad09-17c12ff8a48e tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:02.883165 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-6d2bc690-a6a0-4343-ac9d-f6b5c881e043 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1407 microversion: 2.20 time: 0.209931 Jul 11 14:41:02.898719 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 50/116] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:41:02 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1407 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:41:03.067991 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c620abcc-0e16-489d-ad09-17c12ff8a48e tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:03.076005 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c620abcc-0e16-489d-ad09-17c12ff8a48e tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1361 microversion: 2.1 time: 0.218220 Jul 11 14:41:03.076593 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 67/117] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:02 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1361 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:03.230102 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c3b9f38b-ed3d-4a1f-90d7-5f0551741727 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:03.231371 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c3b9f38b-ed3d-4a1f-90d7-5f0551741727 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:03.233565 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-394eec56-a5ab-43b6-beb4-26c90aacf640 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:03.234250 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-394eec56-a5ab-43b6-beb4-26c90aacf640 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:03.240029 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c3b9f38b-ed3d-4a1f-90d7-5f0551741727 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:03.241136 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-394eec56-a5ab-43b6-beb4-26c90aacf640 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:03.241429 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c3b9f38b-ed3d-4a1f-90d7-5f0551741727 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:03.241596 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-394eec56-a5ab-43b6-beb4-26c90aacf640 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:03.242546 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c3b9f38b-ed3d-4a1f-90d7-5f0551741727 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:03.242666 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-394eec56-a5ab-43b6-beb4-26c90aacf640 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:03.243212 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-394eec56-a5ab-43b6-beb4-26c90aacf640 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:03.243612 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c3b9f38b-ed3d-4a1f-90d7-5f0551741727 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:03.408564 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c3b9f38b-ed3d-4a1f-90d7-5f0551741727 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:03.415969 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c3b9f38b-ed3d-4a1f-90d7-5f0551741727 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1315 microversion: 2.1 time: 0.190723 Jul 11 14:41:03.425162 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 51/118] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:03 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1315 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:03.445309 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-394eec56-a5ab-43b6-beb4-26c90aacf640 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:03.454757 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-394eec56-a5ab-43b6-beb4-26c90aacf640 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1368 microversion: 2.1 time: 0.226329 Jul 11 14:41:03.455671 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 68/119] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:03 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1368 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:03.592346 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-33009536-f596-490c-8cf9-d4f61047ab66 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:03.593041 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-33009536-f596-490c-8cf9-d4f61047ab66 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:03.599746 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-33009536-f596-490c-8cf9-d4f61047ab66 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:03.600199 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-33009536-f596-490c-8cf9-d4f61047ab66 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:03.600973 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-33009536-f596-490c-8cf9-d4f61047ab66 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:03.601592 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-33009536-f596-490c-8cf9-d4f61047ab66 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:03.795523 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-33009536-f596-490c-8cf9-d4f61047ab66 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:03.806349 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-33009536-f596-490c-8cf9-d4f61047ab66 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1322 microversion: 2.1 time: 0.217648 Jul 11 14:41:03.807995 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 52/120] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:03 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1322 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:03.901884 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8ad06aed-8b49-4645-9010-be676e619ec2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:03.906515 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8ad06aed-8b49-4645-9010-be676e619ec2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:03.919554 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8ad06aed-8b49-4645-9010-be676e619ec2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:03.920578 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8ad06aed-8b49-4645-9010-be676e619ec2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:03.921821 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8ad06aed-8b49-4645-9010-be676e619ec2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:03.922814 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8ad06aed-8b49-4645-9010-be676e619ec2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:04.098742 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-43fb4de2-cf8b-4a71-927c-a0d914af18dc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:04.100107 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-43fb4de2-cf8b-4a71-927c-a0d914af18dc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:04.109363 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-43fb4de2-cf8b-4a71-927c-a0d914af18dc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:04.110235 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-43fb4de2-cf8b-4a71-927c-a0d914af18dc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:04.112079 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-43fb4de2-cf8b-4a71-927c-a0d914af18dc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:04.116036 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-43fb4de2-cf8b-4a71-927c-a0d914af18dc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:04.334736 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8ad06aed-8b49-4645-9010-be676e619ec2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:04.369096 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8ad06aed-8b49-4645-9010-be676e619ec2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:04.373056 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8ad06aed-8b49-4645-9010-be676e619ec2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:04.381729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8ad06aed-8b49-4645-9010-be676e619ec2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1407 microversion: 2.20 time: 0.482620 Jul 11 14:41:04.393045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 69/121] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:41:03 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1407 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:41:04.435645 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-27e0ef64-0573-4d6c-ba22-f7ee05fff167 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:04.436277 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-27e0ef64-0573-4d6c-ba22-f7ee05fff167 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:04.448222 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-27e0ef64-0573-4d6c-ba22-f7ee05fff167 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:04.453441 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-27e0ef64-0573-4d6c-ba22-f7ee05fff167 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:04.454889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-27e0ef64-0573-4d6c-ba22-f7ee05fff167 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:04.455655 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-27e0ef64-0573-4d6c-ba22-f7ee05fff167 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:04.503043 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-43fb4de2-cf8b-4a71-927c-a0d914af18dc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:04.513019 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-43fb4de2-cf8b-4a71-927c-a0d914af18dc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1361 microversion: 2.1 time: 0.417902 Jul 11 14:41:04.513821 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 53/122] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:04 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1361 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:04.521201 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-b2fcf266-cefa-4fde-8c64-5699eed165ce tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:04.522104 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-b2fcf266-cefa-4fde-8c64-5699eed165ce tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:04.530309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b2fcf266-cefa-4fde-8c64-5699eed165ce tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:04.530894 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b2fcf266-cefa-4fde-8c64-5699eed165ce tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:04.531999 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b2fcf266-cefa-4fde-8c64-5699eed165ce tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:04.532549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b2fcf266-cefa-4fde-8c64-5699eed165ce tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:04.626188 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-27e0ef64-0573-4d6c-ba22-f7ee05fff167 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:04.634887 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-27e0ef64-0573-4d6c-ba22-f7ee05fff167 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1315 microversion: 2.1 time: 0.204420 Jul 11 14:41:04.635704 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 70/123] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:04 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1315 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:41:04.691024 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-b2fcf266-cefa-4fde-8c64-5699eed165ce tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:04.702495 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-b2fcf266-cefa-4fde-8c64-5699eed165ce tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1368 microversion: 2.1 time: 0.187268 Jul 11 14:41:04.703751 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 54/124] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:04 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1368 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:04.829190 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-4eef9771-b901-47e8-af2e-99141e68dbd6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:04.830690 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-4eef9771-b901-47e8-af2e-99141e68dbd6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:04.837733 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4eef9771-b901-47e8-af2e-99141e68dbd6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:04.838940 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4eef9771-b901-47e8-af2e-99141e68dbd6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:04.839915 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4eef9771-b901-47e8-af2e-99141e68dbd6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:04.840497 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4eef9771-b901-47e8-af2e-99141e68dbd6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:05.009586 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-4eef9771-b901-47e8-af2e-99141e68dbd6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:05.019603 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-4eef9771-b901-47e8-af2e-99141e68dbd6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1322 microversion: 2.1 time: 0.198157 Jul 11 14:41:05.029079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 71/125] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:04 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1322 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:05.057821 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-e4b95434-e954-46c8-bd62-12d484b22bbd req-cf61d773-4e62-4e0f-8f2f-30a55a87d589 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bd855cba-6708-47de-826b-00c5a74b23f2", "name": "network-vif-plugged", "status": "completed", "tag": "8b901616-9eed-4045-b039-9047df0bd8e7"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:41:05.068395 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-e4b95434-e954-46c8-bd62-12d484b22bbd req-cf61d773-4e62-4e0f-8f2f-30a55a87d589 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:05.069120 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-e4b95434-e954-46c8-bd62-12d484b22bbd req-cf61d773-4e62-4e0f-8f2f-30a55a87d589 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:05.084071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-e4b95434-e954-46c8-bd62-12d484b22bbd req-cf61d773-4e62-4e0f-8f2f-30a55a87d589 service nova] Creating event network-vif-plugged:8b901616-9eed-4045-b039-9047df0bd8e7 for instance bd855cba-6708-47de-826b-00c5a74b23f2 on hv-cinder-maste Jul 11 14:41:05.089457 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-e4b95434-e954-46c8-bd62-12d484b22bbd req-cf61d773-4e62-4e0f-8f2f-30a55a87d589 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040737 Jul 11 14:41:05.090067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 55/126] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:41:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:41:05.402130 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-89ad9ff4-5b7d-4f26-9502-8cc9ba23c91f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:05.403139 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-89ad9ff4-5b7d-4f26-9502-8cc9ba23c91f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:05.414114 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-89ad9ff4-5b7d-4f26-9502-8cc9ba23c91f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:05.414790 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-89ad9ff4-5b7d-4f26-9502-8cc9ba23c91f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:05.415690 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-89ad9ff4-5b7d-4f26-9502-8cc9ba23c91f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:05.419105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-89ad9ff4-5b7d-4f26-9502-8cc9ba23c91f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:05.533361 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-258ac6de-cd23-4edb-a752-8fec04d7bdfc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:05.534302 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-258ac6de-cd23-4edb-a752-8fec04d7bdfc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:05.548213 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-258ac6de-cd23-4edb-a752-8fec04d7bdfc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:05.549362 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-258ac6de-cd23-4edb-a752-8fec04d7bdfc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:05.550894 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-258ac6de-cd23-4edb-a752-8fec04d7bdfc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:05.552638 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-258ac6de-cd23-4edb-a752-8fec04d7bdfc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:05.704387 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-89ad9ff4-5b7d-4f26-9502-8cc9ba23c91f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:05.727887 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-89ad9ff4-5b7d-4f26-9502-8cc9ba23c91f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:05.731052 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-89ad9ff4-5b7d-4f26-9502-8cc9ba23c91f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:05.732646 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-89ad9ff4-5b7d-4f26-9502-8cc9ba23c91f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1407 microversion: 2.20 time: 0.335780 Jul 11 14:41:05.740699 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 72/127] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:41:05 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1407 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:41:05.754477 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-0d3f3658-d40f-4661-8e79-cb1666fd79a9 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:05.755199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-0d3f3658-d40f-4661-8e79-cb1666fd79a9 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:05.763450 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0d3f3658-d40f-4661-8e79-cb1666fd79a9 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:05.764009 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0d3f3658-d40f-4661-8e79-cb1666fd79a9 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:05.764872 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0d3f3658-d40f-4661-8e79-cb1666fd79a9 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:05.765394 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0d3f3658-d40f-4661-8e79-cb1666fd79a9 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:05.799199 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-258ac6de-cd23-4edb-a752-8fec04d7bdfc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:05.807395 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-258ac6de-cd23-4edb-a752-8fec04d7bdfc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1361 microversion: 2.1 time: 0.278989 Jul 11 14:41:05.808582 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 56/128] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:05 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1361 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:05.814092 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7b88a990-31b4-4b71-bafb-7091e5e278e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:05.815149 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-7b88a990-31b4-4b71-bafb-7091e5e278e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:05.821691 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7b88a990-31b4-4b71-bafb-7091e5e278e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:05.822179 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7b88a990-31b4-4b71-bafb-7091e5e278e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:05.823886 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7b88a990-31b4-4b71-bafb-7091e5e278e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:05.824459 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7b88a990-31b4-4b71-bafb-7091e5e278e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:05.952787 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0d3f3658-d40f-4661-8e79-cb1666fd79a9 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:05.965491 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-0d3f3658-d40f-4661-8e79-cb1666fd79a9 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1315 microversion: 2.1 time: 0.223654 Jul 11 14:41:05.975706 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 73/129] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:05 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1315 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:05.977771 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7b88a990-31b4-4b71-bafb-7091e5e278e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:05.991197 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-7b88a990-31b4-4b71-bafb-7091e5e278e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1368 microversion: 2.1 time: 0.180904 Jul 11 14:41:05.991597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 57/130] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:05 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1368 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:06.037078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-86bda7d1-4226-406f-acdc-e8c9f59e5aaa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:06.037903 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-86bda7d1-4226-406f-acdc-e8c9f59e5aaa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:06.045002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-86bda7d1-4226-406f-acdc-e8c9f59e5aaa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:06.045451 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-86bda7d1-4226-406f-acdc-e8c9f59e5aaa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:06.046204 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-86bda7d1-4226-406f-acdc-e8c9f59e5aaa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:06.046823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-86bda7d1-4226-406f-acdc-e8c9f59e5aaa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:06.200043 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-86bda7d1-4226-406f-acdc-e8c9f59e5aaa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:06.209969 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-86bda7d1-4226-406f-acdc-e8c9f59e5aaa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1515 microversion: 2.1 time: 0.177252 Jul 11 14:41:06.210510 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 74/131] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:06 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1515 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:06.227683 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8b9bd684-e2ee-4070-8dc3-41ba3cd8c8da tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:41:06.228567 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-8b9bd684-e2ee-4070-8dc3-41ba3cd8c8da tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:06.234708 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8b9bd684-e2ee-4070-8dc3-41ba3cd8c8da tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:06.235162 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8b9bd684-e2ee-4070-8dc3-41ba3cd8c8da tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:06.235926 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8b9bd684-e2ee-4070-8dc3-41ba3cd8c8da tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:06.236367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8b9bd684-e2ee-4070-8dc3-41ba3cd8c8da tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:06.311950 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.objects.instance [None req-8b9bd684-e2ee-4070-8dc3-41ba3cd8c8da tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lazy-loading 'flavor' on Instance uuid cc0591e3-09b9-41cd-b222-f730282f8572 {{(pid=130160) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:41:06.436548 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8b9bd684-e2ee-4070-8dc3-41ba3cd8c8da tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "POST /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572/action" status: 200 len: 32 microversion: 2.1 time: 0.213027 Jul 11 14:41:06.437471 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 58/132] 10.222.0.72 () {60 vars in 1277 bytes} [Mon Jul 11 14:41:06 2022] POST /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572/action => generated 32 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 11 14:41:06.449538 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1c935d37-4b9d-4232-80d9-c500a4b0ad85 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:06.449538 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1c935d37-4b9d-4232-80d9-c500a4b0ad85 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:06.453510 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1c935d37-4b9d-4232-80d9-c500a4b0ad85 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:06.458973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1c935d37-4b9d-4232-80d9-c500a4b0ad85 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:06.458973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1c935d37-4b9d-4232-80d9-c500a4b0ad85 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:06.458973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1c935d37-4b9d-4232-80d9-c500a4b0ad85 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:06.613329 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1c935d37-4b9d-4232-80d9-c500a4b0ad85 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:06.617841 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=100 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:41:06.625422 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1c935d37-4b9d-4232-80d9-c500a4b0ad85 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.182836 Jul 11 14:41:06.626160 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 75/133] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:06 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:41:06.758544 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a2ce63e6-466b-4ba2-b977-767888c1f361 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:06.759808 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a2ce63e6-466b-4ba2-b977-767888c1f361 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:06.768037 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a2ce63e6-466b-4ba2-b977-767888c1f361 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:06.768985 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a2ce63e6-466b-4ba2-b977-767888c1f361 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:06.770631 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a2ce63e6-466b-4ba2-b977-767888c1f361 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:06.771489 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a2ce63e6-466b-4ba2-b977-767888c1f361 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:06.835190 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8abac320-5553-4aad-937a-a1288bffcece tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:06.836322 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8abac320-5553-4aad-937a-a1288bffcece tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:06.862669 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8abac320-5553-4aad-937a-a1288bffcece tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:06.864293 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8abac320-5553-4aad-937a-a1288bffcece tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:06.867039 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8abac320-5553-4aad-937a-a1288bffcece tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:06.867874 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8abac320-5553-4aad-937a-a1288bffcece tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:07.050479 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a2ce63e6-466b-4ba2-b977-767888c1f361 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:07.064024 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a2ce63e6-466b-4ba2-b977-767888c1f361 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:07.071425 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a2ce63e6-466b-4ba2-b977-767888c1f361 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:07.074851 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a2ce63e6-466b-4ba2-b977-767888c1f361 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1407 microversion: 2.20 time: 0.320959 Jul 11 14:41:07.078804 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 59/134] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:41:06 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1407 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:41:07.098330 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9b0cd0b7-c33c-42f3-83e9-7046897f56f4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:07.099144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-9b0cd0b7-c33c-42f3-83e9-7046897f56f4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:07.118494 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9b0cd0b7-c33c-42f3-83e9-7046897f56f4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:07.119342 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9b0cd0b7-c33c-42f3-83e9-7046897f56f4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:07.120150 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9b0cd0b7-c33c-42f3-83e9-7046897f56f4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:07.120586 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9b0cd0b7-c33c-42f3-83e9-7046897f56f4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:07.156494 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8abac320-5553-4aad-937a-a1288bffcece tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:07.166520 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8abac320-5553-4aad-937a-a1288bffcece tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1361 microversion: 2.1 time: 0.335457 Jul 11 14:41:07.167915 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 76/135] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:06 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1361 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:41:07.176852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f3035a1f-4e0f-4b95-bbb1-7001f597d9b4 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:07.177795 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f3035a1f-4e0f-4b95-bbb1-7001f597d9b4 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:07.184161 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f3035a1f-4e0f-4b95-bbb1-7001f597d9b4 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:07.184633 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f3035a1f-4e0f-4b95-bbb1-7001f597d9b4 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:07.185410 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f3035a1f-4e0f-4b95-bbb1-7001f597d9b4 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:07.186042 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f3035a1f-4e0f-4b95-bbb1-7001f597d9b4 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:07.621161 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f3035a1f-4e0f-4b95-bbb1-7001f597d9b4 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:07.633604 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f3035a1f-4e0f-4b95-bbb1-7001f597d9b4 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1368 microversion: 2.1 time: 0.463833 Jul 11 14:41:07.634845 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 77/136] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:07 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1368 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:07.648683 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-26a5f606-6b92-4a6a-9033-5f6485949f5e req-31c84f05-d5e5-4d95-b54f-4113c7c96818 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a3e173ea-1d52-48e5-a5d1-89195af310c7", "name": "network-vif-plugged", "status": "completed", "tag": "5f8fa020-0a9e-499b-a83e-8e2d443ee007"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:41:07.660271 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-26a5f606-6b92-4a6a-9033-5f6485949f5e req-31c84f05-d5e5-4d95-b54f-4113c7c96818 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:07.660724 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-26a5f606-6b92-4a6a-9033-5f6485949f5e req-31c84f05-d5e5-4d95-b54f-4113c7c96818 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:07.678763 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-26a5f606-6b92-4a6a-9033-5f6485949f5e req-31c84f05-d5e5-4d95-b54f-4113c7c96818 service nova] Creating event network-vif-plugged:5f8fa020-0a9e-499b-a83e-8e2d443ee007 for instance a3e173ea-1d52-48e5-a5d1-89195af310c7 on hv-cinder-maste Jul 11 14:41:07.681939 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-9b0cd0b7-c33c-42f3-83e9-7046897f56f4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:07.687117 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-26a5f606-6b92-4a6a-9033-5f6485949f5e req-31c84f05-d5e5-4d95-b54f-4113c7c96818 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049314 Jul 11 14:41:07.688214 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 78/137] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:41:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:41:07.692190 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9b0cd0b7-c33c-42f3-83e9-7046897f56f4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1315 microversion: 2.1 time: 0.610339 Jul 11 14:41:07.693450 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 60/138] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:07 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1315 bytes in 614 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:07.696213 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-bccc8275-a324-4baf-b7bb-c535aac2fee0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:07.697756 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-bccc8275-a324-4baf-b7bb-c535aac2fee0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:07.710101 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bccc8275-a324-4baf-b7bb-c535aac2fee0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:07.711224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bccc8275-a324-4baf-b7bb-c535aac2fee0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:07.712018 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bccc8275-a324-4baf-b7bb-c535aac2fee0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:07.712777 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bccc8275-a324-4baf-b7bb-c535aac2fee0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:08.098780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-fd2643d7-1947-4ed6-bd1f-da3a3131d64d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:08.100397 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-fd2643d7-1947-4ed6-bd1f-da3a3131d64d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:08.111297 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd2643d7-1947-4ed6-bd1f-da3a3131d64d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:08.111904 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd2643d7-1947-4ed6-bd1f-da3a3131d64d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:08.113020 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd2643d7-1947-4ed6-bd1f-da3a3131d64d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:08.113826 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd2643d7-1947-4ed6-bd1f-da3a3131d64d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:08.157740 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-bccc8275-a324-4baf-b7bb-c535aac2fee0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:08.166023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-bccc8275-a324-4baf-b7bb-c535aac2fee0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.475502 Jul 11 14:41:08.167341 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 79/139] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:07 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:08.184776 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-86cd3523-5827-4010-95ed-27bf393e362b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:08.185733 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-86cd3523-5827-4010-95ed-27bf393e362b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:08.193559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-86cd3523-5827-4010-95ed-27bf393e362b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:08.194354 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-86cd3523-5827-4010-95ed-27bf393e362b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:08.195382 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-86cd3523-5827-4010-95ed-27bf393e362b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:08.196040 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-86cd3523-5827-4010-95ed-27bf393e362b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:08.361746 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-fd2643d7-1947-4ed6-bd1f-da3a3131d64d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:08.373708 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-fd2643d7-1947-4ed6-bd1f-da3a3131d64d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:08.375837 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-fd2643d7-1947-4ed6-bd1f-da3a3131d64d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:08.381919 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-fd2643d7-1947-4ed6-bd1f-da3a3131d64d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1407 microversion: 2.20 time: 0.283417 Jul 11 14:41:08.381919 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 61/140] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:41:08 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1407 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:41:08.449571 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-86cd3523-5827-4010-95ed-27bf393e362b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:08.458565 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-86cd3523-5827-4010-95ed-27bf393e362b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1361 microversion: 2.1 time: 0.278041 Jul 11 14:41:08.459279 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 80/141] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:08 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1361 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:41:08.653321 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-309572c6-f021-4f55-8c1e-d161dbc8ea07 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:08.654074 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-309572c6-f021-4f55-8c1e-d161dbc8ea07 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:08.673445 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-309572c6-f021-4f55-8c1e-d161dbc8ea07 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:08.674154 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-309572c6-f021-4f55-8c1e-d161dbc8ea07 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:08.675310 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-309572c6-f021-4f55-8c1e-d161dbc8ea07 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:08.675892 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-309572c6-f021-4f55-8c1e-d161dbc8ea07 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:08.714683 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f546d9ca-6b8f-4661-bc36-62feb26f08cc tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:08.715725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f546d9ca-6b8f-4661-bc36-62feb26f08cc tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:08.726317 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f546d9ca-6b8f-4661-bc36-62feb26f08cc tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:08.726848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f546d9ca-6b8f-4661-bc36-62feb26f08cc tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:08.735281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f546d9ca-6b8f-4661-bc36-62feb26f08cc tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:08.735843 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f546d9ca-6b8f-4661-bc36-62feb26f08cc tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:08.977864 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-309572c6-f021-4f55-8c1e-d161dbc8ea07 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:08.988929 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f546d9ca-6b8f-4661-bc36-62feb26f08cc tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:08.991767 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-309572c6-f021-4f55-8c1e-d161dbc8ea07 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1368 microversion: 2.1 time: 0.343303 Jul 11 14:41:08.993928 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 62/142] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:08 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1368 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:09.002852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f546d9ca-6b8f-4661-bc36-62feb26f08cc tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1315 microversion: 2.1 time: 0.294655 Jul 11 14:41:09.003517 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 81/143] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:08 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1315 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:09.185300 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-caf65987-0fd9-4995-ac30-d77f8ece8780 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:09.185929 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-caf65987-0fd9-4995-ac30-d77f8ece8780 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:09.200031 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-caf65987-0fd9-4995-ac30-d77f8ece8780 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:09.200899 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-caf65987-0fd9-4995-ac30-d77f8ece8780 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:09.201721 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-caf65987-0fd9-4995-ac30-d77f8ece8780 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:09.202152 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-caf65987-0fd9-4995-ac30-d77f8ece8780 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:09.208072 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:41:09.395068 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-17939c30-e742-4711-9726-ff0ed29caefa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:09.396556 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-17939c30-e742-4711-9726-ff0ed29caefa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:09.409682 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-17939c30-e742-4711-9726-ff0ed29caefa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:09.412680 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-17939c30-e742-4711-9726-ff0ed29caefa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:09.413978 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-17939c30-e742-4711-9726-ff0ed29caefa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:09.415560 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-17939c30-e742-4711-9726-ff0ed29caefa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:09.486585 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-caf65987-0fd9-4995-ac30-d77f8ece8780 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:09.501520 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-caf65987-0fd9-4995-ac30-d77f8ece8780 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.317906 Jul 11 14:41:09.502538 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 63/144] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:09 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:09.510385 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9c3bf4e2-5c84-4944-a117-76221b1aaee2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:09.516177 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-9c3bf4e2-5c84-4944-a117-76221b1aaee2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:09.530298 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9c3bf4e2-5c84-4944-a117-76221b1aaee2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:09.531112 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9c3bf4e2-5c84-4944-a117-76221b1aaee2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:09.532119 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9c3bf4e2-5c84-4944-a117-76221b1aaee2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:09.532783 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9c3bf4e2-5c84-4944-a117-76221b1aaee2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:09.804229 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-9c3bf4e2-5c84-4944-a117-76221b1aaee2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:09.839090 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9c3bf4e2-5c84-4944-a117-76221b1aaee2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1361 microversion: 2.1 time: 0.334247 Jul 11 14:41:09.840839 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 64/145] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:09 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1361 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:09.846833 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-64776210-3bfa-492b-bec6-d506ebfaeef0 req-bb0ec682-109a-440e-ae03-99dc795125b5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "13da005a-e085-43aa-8a18-1fe85e4e95af", "name": "network-vif-plugged", "status": "completed", "tag": "7b725b13-befd-4a14-98cc-9d0e599f5e17"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:41:09.864428 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-64776210-3bfa-492b-bec6-d506ebfaeef0 req-bb0ec682-109a-440e-ae03-99dc795125b5 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:09.867220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-64776210-3bfa-492b-bec6-d506ebfaeef0 req-bb0ec682-109a-440e-ae03-99dc795125b5 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:09.887194 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-64776210-3bfa-492b-bec6-d506ebfaeef0 req-bb0ec682-109a-440e-ae03-99dc795125b5 service nova] Creating event network-vif-plugged:7b725b13-befd-4a14-98cc-9d0e599f5e17 for instance 13da005a-e085-43aa-8a18-1fe85e4e95af on hv-cinder-maste Jul 11 14:41:09.899797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-64776210-3bfa-492b-bec6-d506ebfaeef0 req-bb0ec682-109a-440e-ae03-99dc795125b5 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056965 Jul 11 14:41:09.901413 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 65/146] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:41:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:41:09.935315 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-17939c30-e742-4711-9726-ff0ed29caefa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:09.945325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-17939c30-e742-4711-9726-ff0ed29caefa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:09.947289 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-17939c30-e742-4711-9726-ff0ed29caefa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:09.949469 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-17939c30-e742-4711-9726-ff0ed29caefa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1407 microversion: 2.20 time: 0.558504 Jul 11 14:41:09.954986 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 82/147] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:41:09 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1407 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:41:10.018919 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-31dd9a50-7b22-4eaa-9cf5-c9bb1fb6ea88 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:10.020217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-31dd9a50-7b22-4eaa-9cf5-c9bb1fb6ea88 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:10.030173 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-31dd9a50-7b22-4eaa-9cf5-c9bb1fb6ea88 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:10.033287 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1373fa4c-2ac9-4e3b-9c94-8358b048463f tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:10.033904 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1373fa4c-2ac9-4e3b-9c94-8358b048463f tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:10.035180 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-31dd9a50-7b22-4eaa-9cf5-c9bb1fb6ea88 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:10.036018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-31dd9a50-7b22-4eaa-9cf5-c9bb1fb6ea88 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:10.036462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-31dd9a50-7b22-4eaa-9cf5-c9bb1fb6ea88 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:10.041800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1373fa4c-2ac9-4e3b-9c94-8358b048463f tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:10.042649 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1373fa4c-2ac9-4e3b-9c94-8358b048463f tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:10.043618 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1373fa4c-2ac9-4e3b-9c94-8358b048463f tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:10.044265 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1373fa4c-2ac9-4e3b-9c94-8358b048463f tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:10.326368 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-31dd9a50-7b22-4eaa-9cf5-c9bb1fb6ea88 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:10.335885 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-31dd9a50-7b22-4eaa-9cf5-c9bb1fb6ea88 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1558 microversion: 2.1 time: 0.328875 Jul 11 14:41:10.348151 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 66/148] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:10 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1558 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:10.354820 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-0fbf9202-ccef-4e51-aaf7-4b4453def900 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:10.355591 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-0fbf9202-ccef-4e51-aaf7-4b4453def900 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:10.362845 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0fbf9202-ccef-4e51-aaf7-4b4453def900 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:10.363692 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0fbf9202-ccef-4e51-aaf7-4b4453def900 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:10.364787 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0fbf9202-ccef-4e51-aaf7-4b4453def900 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:10.365421 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0fbf9202-ccef-4e51-aaf7-4b4453def900 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:10.440607 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1373fa4c-2ac9-4e3b-9c94-8358b048463f tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:10.455934 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1373fa4c-2ac9-4e3b-9c94-8358b048463f tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1315 microversion: 2.1 time: 0.433573 Jul 11 14:41:10.456796 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 83/149] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:10 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1315 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:10.486375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-0fbf9202-ccef-4e51-aaf7-4b4453def900 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.136529 Jul 11 14:41:10.487021 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 67/150] 10.222.0.72 () {58 vars in 1274 bytes} [Mon Jul 11 14:41:10 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-interface => generated 285 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:41:10.525101 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1b3a9035-cfa7-4657-9ec6-eb19abc40b8d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:10.526306 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1b3a9035-cfa7-4657-9ec6-eb19abc40b8d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:10.529969 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=100 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:41:10.544818 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1b3a9035-cfa7-4657-9ec6-eb19abc40b8d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:10.545289 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1b3a9035-cfa7-4657-9ec6-eb19abc40b8d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:10.546086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1b3a9035-cfa7-4657-9ec6-eb19abc40b8d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:10.546555 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1b3a9035-cfa7-4657-9ec6-eb19abc40b8d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:10.858325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-25b4ced6-8587-43f9-b4b3-a1c16004db48 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:10.858985 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-25b4ced6-8587-43f9-b4b3-a1c16004db48 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:10.867888 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-25b4ced6-8587-43f9-b4b3-a1c16004db48 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:10.868351 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-25b4ced6-8587-43f9-b4b3-a1c16004db48 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:10.869147 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-25b4ced6-8587-43f9-b4b3-a1c16004db48 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:10.869630 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-25b4ced6-8587-43f9-b4b3-a1c16004db48 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:10.954471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1b3a9035-cfa7-4657-9ec6-eb19abc40b8d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:10.969314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1b3a9035-cfa7-4657-9ec6-eb19abc40b8d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.448197 Jul 11 14:41:10.974616 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 84/151] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:10 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:10.984276 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c4839558-12dc-4bc2-8245-0ea91455fa32 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:10.985041 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c4839558-12dc-4bc2-8245-0ea91455fa32 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:10.998718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c4839558-12dc-4bc2-8245-0ea91455fa32 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:11.004384 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c4839558-12dc-4bc2-8245-0ea91455fa32 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:11.005535 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c4839558-12dc-4bc2-8245-0ea91455fa32 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:11.005996 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c4839558-12dc-4bc2-8245-0ea91455fa32 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:11.201939 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c4839558-12dc-4bc2-8245-0ea91455fa32 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:11.213456 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c4839558-12dc-4bc2-8245-0ea91455fa32 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:11.215533 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c4839558-12dc-4bc2-8245-0ea91455fa32 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:11.216884 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c4839558-12dc-4bc2-8245-0ea91455fa32 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1407 microversion: 2.20 time: 0.244766 Jul 11 14:41:11.217871 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 85/152] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:41:10 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1407 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:41:11.382467 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-25b4ced6-8587-43f9-b4b3-a1c16004db48 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:11.391548 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-25b4ced6-8587-43f9-b4b3-a1c16004db48 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1361 microversion: 2.1 time: 0.537737 Jul 11 14:41:11.393191 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 68/153] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:10 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1361 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:11.473869 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-88414c58-3891-4d6c-b389-4debb901f6c7 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:11.475321 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-88414c58-3891-4d6c-b389-4debb901f6c7 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:11.483070 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-88414c58-3891-4d6c-b389-4debb901f6c7 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:11.484016 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-88414c58-3891-4d6c-b389-4debb901f6c7 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:11.485423 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-88414c58-3891-4d6c-b389-4debb901f6c7 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:11.486521 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-88414c58-3891-4d6c-b389-4debb901f6c7 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:11.880576 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-88414c58-3891-4d6c-b389-4debb901f6c7 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:11.890833 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-88414c58-3891-4d6c-b389-4debb901f6c7 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1500 microversion: 2.1 time: 0.421130 Jul 11 14:41:11.892394 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 86/154] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:11 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1500 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:11.930944 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-85cf9d49-814d-4033-9aa3-27885806179b req-9d11ff61-ad57-492f-9e68-e7cd60fb09d1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "62a90c58-0027-47e8-8909-44512060e08d", "name": "network-vif-plugged", "status": "completed", "tag": "edfcc9e3-9cd5-4b35-adc9-68a5f06aeb87"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:41:11.952551 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=100 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:41:11.972810 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-85cf9d49-814d-4033-9aa3-27885806179b req-9d11ff61-ad57-492f-9e68-e7cd60fb09d1 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:11.973573 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-85cf9d49-814d-4033-9aa3-27885806179b req-9d11ff61-ad57-492f-9e68-e7cd60fb09d1 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:11.992407 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-43d2391e-7c6f-4399-bbb7-c0b67cb43fed tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:11.993507 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-43d2391e-7c6f-4399-bbb7-c0b67cb43fed tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:12.024713 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-43d2391e-7c6f-4399-bbb7-c0b67cb43fed tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:12.026798 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-43d2391e-7c6f-4399-bbb7-c0b67cb43fed tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:12.032058 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-43d2391e-7c6f-4399-bbb7-c0b67cb43fed tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:12.033162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-43d2391e-7c6f-4399-bbb7-c0b67cb43fed tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:12.033735 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-85cf9d49-814d-4033-9aa3-27885806179b req-9d11ff61-ad57-492f-9e68-e7cd60fb09d1 service nova] Creating event network-vif-plugged:edfcc9e3-9cd5-4b35-adc9-68a5f06aeb87 for instance 62a90c58-0027-47e8-8909-44512060e08d on hv-cinder-maste Jul 11 14:41:12.046073 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-85cf9d49-814d-4033-9aa3-27885806179b req-9d11ff61-ad57-492f-9e68-e7cd60fb09d1 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.120619 Jul 11 14:41:12.047810 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 69/155] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:41:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:41:12.242198 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-5838be65-1b47-4d2d-8bbe-6812c7760a66 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:12.243497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-5838be65-1b47-4d2d-8bbe-6812c7760a66 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:12.290994 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5838be65-1b47-4d2d-8bbe-6812c7760a66 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:12.292832 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5838be65-1b47-4d2d-8bbe-6812c7760a66 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:12.297916 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5838be65-1b47-4d2d-8bbe-6812c7760a66 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:12.299132 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5838be65-1b47-4d2d-8bbe-6812c7760a66 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:12.384516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-43d2391e-7c6f-4399-bbb7-c0b67cb43fed tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:12.405801 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-43d2391e-7c6f-4399-bbb7-c0b67cb43fed tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.419235 Jul 11 14:41:12.407879 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 87/156] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:11 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:12.436464 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-684fdc8e-cddd-441e-a323-7589e69c9366 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:12.437107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-684fdc8e-cddd-441e-a323-7589e69c9366 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:12.456567 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-684fdc8e-cddd-441e-a323-7589e69c9366 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:12.456974 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-684fdc8e-cddd-441e-a323-7589e69c9366 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:12.467717 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-684fdc8e-cddd-441e-a323-7589e69c9366 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:12.467717 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-684fdc8e-cddd-441e-a323-7589e69c9366 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:12.620410 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5838be65-1b47-4d2d-8bbe-6812c7760a66 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:12.635799 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5838be65-1b47-4d2d-8bbe-6812c7760a66 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:12.642572 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5838be65-1b47-4d2d-8bbe-6812c7760a66 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:12.647387 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-5838be65-1b47-4d2d-8bbe-6812c7760a66 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1407 microversion: 2.20 time: 0.414270 Jul 11 14:41:12.651666 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 70/157] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:41:12 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1407 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:41:12.742870 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-684fdc8e-cddd-441e-a323-7589e69c9366 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:12.751478 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-684fdc8e-cddd-441e-a323-7589e69c9366 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1361 microversion: 2.1 time: 0.339502 Jul 11 14:41:12.760225 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 88/158] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:12 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1361 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:13.426468 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8c22a820-38e5-4700-8f1c-1055c69c55b9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:13.427200 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-8c22a820-38e5-4700-8f1c-1055c69c55b9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:13.437600 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8c22a820-38e5-4700-8f1c-1055c69c55b9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:13.442134 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8c22a820-38e5-4700-8f1c-1055c69c55b9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:13.448968 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8c22a820-38e5-4700-8f1c-1055c69c55b9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:13.449768 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8c22a820-38e5-4700-8f1c-1055c69c55b9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:13.466493 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-46d7e0d9-6664-4590-a70c-3bb5235bee29 req-98d9b5fc-d119-4abf-a297-da969876a9a2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bd855cba-6708-47de-826b-00c5a74b23f2", "tag": "8b901616-9eed-4045-b039-9047df0bd8e7"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:41:13.484288 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-46d7e0d9-6664-4590-a70c-3bb5235bee29 req-98d9b5fc-d119-4abf-a297-da969876a9a2 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:13.494284 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-46d7e0d9-6664-4590-a70c-3bb5235bee29 req-98d9b5fc-d119-4abf-a297-da969876a9a2 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:13.515671 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-46d7e0d9-6664-4590-a70c-3bb5235bee29 req-98d9b5fc-d119-4abf-a297-da969876a9a2 service nova] Creating event network-changed:8b901616-9eed-4045-b039-9047df0bd8e7 for instance bd855cba-6708-47de-826b-00c5a74b23f2 on hv-cinder-maste Jul 11 14:41:13.522154 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-46d7e0d9-6664-4590-a70c-3bb5235bee29 req-98d9b5fc-d119-4abf-a297-da969876a9a2 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059946 Jul 11 14:41:13.523269 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 89/159] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:41:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:41:13.672221 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-7c43a481-7352-452c-bc58-c4f574624143 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:13.672221 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-7c43a481-7352-452c-bc58-c4f574624143 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:13.677739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7c43a481-7352-452c-bc58-c4f574624143 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:13.683110 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7c43a481-7352-452c-bc58-c4f574624143 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:13.683943 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7c43a481-7352-452c-bc58-c4f574624143 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:13.689995 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7c43a481-7352-452c-bc58-c4f574624143 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:13.857235 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8c22a820-38e5-4700-8f1c-1055c69c55b9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:13.871754 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8c22a820-38e5-4700-8f1c-1055c69c55b9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.449383 Jul 11 14:41:13.872532 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 71/160] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:13 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:13.879885 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-55004e6c-8fcc-41a9-a4bc-8c5d119680d7 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ef3bdf46-21c6-48a2-aa2e-103a53608a4d"}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:41:13.881203 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-55004e6c-8fcc-41a9-a4bc-8c5d119680d7 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:13.899873 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-55004e6c-8fcc-41a9-a4bc-8c5d119680d7 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:13.902272 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-55004e6c-8fcc-41a9-a4bc-8c5d119680d7 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:13.905665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-55004e6c-8fcc-41a9-a4bc-8c5d119680d7 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:13.907903 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-55004e6c-8fcc-41a9-a4bc-8c5d119680d7 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:14.025460 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7c43a481-7352-452c-bc58-c4f574624143 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:14.046153 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7c43a481-7352-452c-bc58-c4f574624143 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:14.051801 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7c43a481-7352-452c-bc58-c4f574624143 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:14.060252 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-7c43a481-7352-452c-bc58-c4f574624143 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1598 microversion: 2.20 time: 0.391155 Jul 11 14:41:14.060922 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 90/161] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:41:13 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1598 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:41:14.070814 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-29c0dac4-d251-4cc7-93d3-f66a8895df42 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:14.074332 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-29c0dac4-d251-4cc7-93d3-f66a8895df42 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:14.083962 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-29c0dac4-d251-4cc7-93d3-f66a8895df42 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:14.084654 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-29c0dac4-d251-4cc7-93d3-f66a8895df42 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:14.086228 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-29c0dac4-d251-4cc7-93d3-f66a8895df42 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:14.086863 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-29c0dac4-d251-4cc7-93d3-f66a8895df42 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:14.347668 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-29c0dac4-d251-4cc7-93d3-f66a8895df42 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:14.361750 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-29c0dac4-d251-4cc7-93d3-f66a8895df42 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1361 microversion: 2.1 time: 0.299484 Jul 11 14:41:14.363847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 91/162] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:14 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1361 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:14.378025 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e351f921-9e8a-4131-8415-672cd953cc0d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:14.378859 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e351f921-9e8a-4131-8415-672cd953cc0d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:14.394229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e351f921-9e8a-4131-8415-672cd953cc0d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:14.398837 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e351f921-9e8a-4131-8415-672cd953cc0d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:14.405140 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e351f921-9e8a-4131-8415-672cd953cc0d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:14.405978 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e351f921-9e8a-4131-8415-672cd953cc0d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:14.454689 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-55004e6c-8fcc-41a9-a4bc-8c5d119680d7 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "POST /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.578543 Jul 11 14:41:14.456531 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 72/163] 10.222.0.72 () {60 vars in 1322 bytes} [Mon Jul 11 14:41:13 2022] POST /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7/os-volume_attachments => generated 194 bytes in 582 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:41:14.619067 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e351f921-9e8a-4131-8415-672cd953cc0d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.253273 Jul 11 14:41:14.619705 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 92/164] 10.222.0.72 () {60 vars in 1315 bytes} [Mon Jul 11 14:41:14 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/os-interface => generated 284 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:41:14.897646 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3822d5ae-f990-4601-8bcf-7ad7877cd852 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:14.899194 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-3822d5ae-f990-4601-8bcf-7ad7877cd852 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:14.920725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3822d5ae-f990-4601-8bcf-7ad7877cd852 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:14.922605 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3822d5ae-f990-4601-8bcf-7ad7877cd852 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:14.923683 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3822d5ae-f990-4601-8bcf-7ad7877cd852 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:14.925012 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3822d5ae-f990-4601-8bcf-7ad7877cd852 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:15.380753 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d6031f7a-2ae4-4141-8334-a77aa6febdfb tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:15.381376 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d6031f7a-2ae4-4141-8334-a77aa6febdfb tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:15.400125 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d6031f7a-2ae4-4141-8334-a77aa6febdfb tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:15.401899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d6031f7a-2ae4-4141-8334-a77aa6febdfb tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:15.407635 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-3822d5ae-f990-4601-8bcf-7ad7877cd852 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:15.407794 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d6031f7a-2ae4-4141-8334-a77aa6febdfb tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:15.408490 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d6031f7a-2ae4-4141-8334-a77aa6febdfb tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:15.419523 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-3822d5ae-f990-4601-8bcf-7ad7877cd852 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.531631 Jul 11 14:41:15.420345 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 73/165] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:14 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 533 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:15.686858 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d6031f7a-2ae4-4141-8334-a77aa6febdfb tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:15.696234 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-d6031f7a-2ae4-4141-8334-a77aa6febdfb tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1548 microversion: 2.1 time: 0.319589 Jul 11 14:41:15.700747 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 93/166] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:15 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1548 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:41:15.719098 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-ef7187fa-7f80-4d12-8776-484df3b7b29b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:15.719596 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-ef7187fa-7f80-4d12-8776-484df3b7b29b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:15.734407 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ef7187fa-7f80-4d12-8776-484df3b7b29b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:15.736602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ef7187fa-7f80-4d12-8776-484df3b7b29b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:15.739423 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ef7187fa-7f80-4d12-8776-484df3b7b29b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:15.741902 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ef7187fa-7f80-4d12-8776-484df3b7b29b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:16.058204 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-ef7187fa-7f80-4d12-8776-484df3b7b29b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.343134 Jul 11 14:41:16.070770 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 74/167] 10.222.0.72 () {58 vars in 1274 bytes} [Mon Jul 11 14:41:15 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d/os-interface => generated 285 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:41:16.437227 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-ba32ed90-62f7-4650-9dd1-d0bdf8d07833 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:16.438537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-ba32ed90-62f7-4650-9dd1-d0bdf8d07833 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:16.444864 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ba32ed90-62f7-4650-9dd1-d0bdf8d07833 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:16.445319 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ba32ed90-62f7-4650-9dd1-d0bdf8d07833 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:16.446398 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ba32ed90-62f7-4650-9dd1-d0bdf8d07833 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:16.446963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ba32ed90-62f7-4650-9dd1-d0bdf8d07833 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:16.451828 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=100 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:41:16.787434 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-ba32ed90-62f7-4650-9dd1-d0bdf8d07833 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:16.798078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-ba32ed90-62f7-4650-9dd1-d0bdf8d07833 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.364576 Jul 11 14:41:16.799311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 94/168] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:16 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:16.948059 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-7c71be7d-d6cc-480a-b63e-e763ad9447cd req-6203ec6e-7657-47fb-9cbc-f5dee1363943 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "13da005a-e085-43aa-8a18-1fe85e4e95af", "tag": "7b725b13-befd-4a14-98cc-9d0e599f5e17"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:41:16.958623 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-7c71be7d-d6cc-480a-b63e-e763ad9447cd req-6203ec6e-7657-47fb-9cbc-f5dee1363943 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:16.959771 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-7c71be7d-d6cc-480a-b63e-e763ad9447cd req-6203ec6e-7657-47fb-9cbc-f5dee1363943 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:16.975862 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-7c71be7d-d6cc-480a-b63e-e763ad9447cd req-6203ec6e-7657-47fb-9cbc-f5dee1363943 service nova] Creating event network-changed:7b725b13-befd-4a14-98cc-9d0e599f5e17 for instance 13da005a-e085-43aa-8a18-1fe85e4e95af on hv-cinder-maste Jul 11 14:41:16.983523 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-7c71be7d-d6cc-480a-b63e-e763ad9447cd req-6203ec6e-7657-47fb-9cbc-f5dee1363943 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041473 Jul 11 14:41:16.984745 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 75/169] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:41:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:41:17.815846 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-5ea79f7a-9b62-4534-8aee-bcc078e60a36 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:17.817061 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-5ea79f7a-9b62-4534-8aee-bcc078e60a36 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:17.832569 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5ea79f7a-9b62-4534-8aee-bcc078e60a36 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:17.835515 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5ea79f7a-9b62-4534-8aee-bcc078e60a36 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:17.836778 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5ea79f7a-9b62-4534-8aee-bcc078e60a36 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:17.837948 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5ea79f7a-9b62-4534-8aee-bcc078e60a36 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:18.123635 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-5ea79f7a-9b62-4534-8aee-bcc078e60a36 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:18.133045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-5ea79f7a-9b62-4534-8aee-bcc078e60a36 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.320372 Jul 11 14:41:18.133658 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 95/170] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:17 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:18.234392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-af17f959-a909-43ac-911c-d7ad2c6fac7b req-7083ce89-8b40-4b75-ac4e-bdd7e9a83d42 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "62a90c58-0027-47e8-8909-44512060e08d", "tag": "edfcc9e3-9cd5-4b35-adc9-68a5f06aeb87"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:41:18.248107 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-af17f959-a909-43ac-911c-d7ad2c6fac7b req-7083ce89-8b40-4b75-ac4e-bdd7e9a83d42 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:18.248554 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-af17f959-a909-43ac-911c-d7ad2c6fac7b req-7083ce89-8b40-4b75-ac4e-bdd7e9a83d42 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:18.286891 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-af17f959-a909-43ac-911c-d7ad2c6fac7b req-7083ce89-8b40-4b75-ac4e-bdd7e9a83d42 service nova] Creating event network-changed:edfcc9e3-9cd5-4b35-adc9-68a5f06aeb87 for instance 62a90c58-0027-47e8-8909-44512060e08d on hv-cinder-maste Jul 11 14:41:18.295657 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-af17f959-a909-43ac-911c-d7ad2c6fac7b req-7083ce89-8b40-4b75-ac4e-bdd7e9a83d42 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065457 Jul 11 14:41:18.296367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 76/171] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:41:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:41:19.149931 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d5367461-0732-4c14-9028-d6e89c1c4718 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:19.151121 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d5367461-0732-4c14-9028-d6e89c1c4718 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:19.164448 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d5367461-0732-4c14-9028-d6e89c1c4718 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:19.164916 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d5367461-0732-4c14-9028-d6e89c1c4718 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:19.165706 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d5367461-0732-4c14-9028-d6e89c1c4718 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:19.166703 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d5367461-0732-4c14-9028-d6e89c1c4718 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:19.451401 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d5367461-0732-4c14-9028-d6e89c1c4718 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:19.464196 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-d5367461-0732-4c14-9028-d6e89c1c4718 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.318833 Jul 11 14:41:19.468166 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 96/172] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:19 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:20.483331 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-ffb6ec04-41ff-4737-b3a5-95b838ddd302 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:20.483994 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-ffb6ec04-41ff-4737-b3a5-95b838ddd302 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:20.495240 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ffb6ec04-41ff-4737-b3a5-95b838ddd302 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:20.496708 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ffb6ec04-41ff-4737-b3a5-95b838ddd302 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:20.497725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ffb6ec04-41ff-4737-b3a5-95b838ddd302 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:20.498250 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ffb6ec04-41ff-4737-b3a5-95b838ddd302 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:20.658337 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-ffb6ec04-41ff-4737-b3a5-95b838ddd302 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:20.670169 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-ffb6ec04-41ff-4737-b3a5-95b838ddd302 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.192575 Jul 11 14:41:20.679324 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 77/173] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:20 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:21.688958 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9cd2c996-ff0a-4c06-af03-326486185790 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:21.689833 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9cd2c996-ff0a-4c06-af03-326486185790 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:21.708754 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9cd2c996-ff0a-4c06-af03-326486185790 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:21.709256 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9cd2c996-ff0a-4c06-af03-326486185790 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:21.710046 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9cd2c996-ff0a-4c06-af03-326486185790 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:21.710560 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9cd2c996-ff0a-4c06-af03-326486185790 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:21.879604 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9cd2c996-ff0a-4c06-af03-326486185790 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:21.889843 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9cd2c996-ff0a-4c06-af03-326486185790 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.204717 Jul 11 14:41:21.890633 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 97/174] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:21 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:22.909631 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-bb4bf33e-f9b4-4712-8ba2-bfe4c563e54e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:22.910396 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-bb4bf33e-f9b4-4712-8ba2-bfe4c563e54e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:22.918799 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bb4bf33e-f9b4-4712-8ba2-bfe4c563e54e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:22.919588 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bb4bf33e-f9b4-4712-8ba2-bfe4c563e54e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:22.920577 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bb4bf33e-f9b4-4712-8ba2-bfe4c563e54e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:22.921293 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bb4bf33e-f9b4-4712-8ba2-bfe4c563e54e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:23.143497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-bb4bf33e-f9b4-4712-8ba2-bfe4c563e54e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:23.152861 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-bb4bf33e-f9b4-4712-8ba2-bfe4c563e54e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.248874 Jul 11 14:41:23.154332 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 78/175] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:22 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:41:24.171019 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e42f078a-7762-4124-90ee-a95b468287d2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:24.172249 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e42f078a-7762-4124-90ee-a95b468287d2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:24.180600 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e42f078a-7762-4124-90ee-a95b468287d2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:24.181182 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e42f078a-7762-4124-90ee-a95b468287d2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:24.182743 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e42f078a-7762-4124-90ee-a95b468287d2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:24.183434 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e42f078a-7762-4124-90ee-a95b468287d2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:24.419421 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e42f078a-7762-4124-90ee-a95b468287d2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:24.434497 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e42f078a-7762-4124-90ee-a95b468287d2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.265742 Jul 11 14:41:24.437263 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 98/176] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:24 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:25.451929 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9ecd773f-177f-4df0-969a-dbb2ead8e80d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:25.452530 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-9ecd773f-177f-4df0-969a-dbb2ead8e80d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:25.456561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-01277c53-d0be-465b-bb36-0b5e0e2bcbcb tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:25.457063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-01277c53-d0be-465b-bb36-0b5e0e2bcbcb tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:25.460266 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9ecd773f-177f-4df0-969a-dbb2ead8e80d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:25.460959 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9ecd773f-177f-4df0-969a-dbb2ead8e80d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:25.461798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9ecd773f-177f-4df0-969a-dbb2ead8e80d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:25.462350 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9ecd773f-177f-4df0-969a-dbb2ead8e80d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:25.465332 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-01277c53-d0be-465b-bb36-0b5e0e2bcbcb tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:25.466940 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-01277c53-d0be-465b-bb36-0b5e0e2bcbcb tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:25.469565 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-01277c53-d0be-465b-bb36-0b5e0e2bcbcb tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:25.470239 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-01277c53-d0be-465b-bb36-0b5e0e2bcbcb tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:25.497828 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-01277c53-d0be-465b-bb36-0b5e0e2bcbcb tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Going to try to terminate instance {{(pid=130159) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:41:25.621446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-01277c53-d0be-465b-bb36-0b5e0e2bcbcb tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "DELETE /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 204 len: 0 microversion: 2.1 time: 0.168647 Jul 11 14:41:25.622120 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 99/177] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:41:25 2022] DELETE /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 11 14:41:25.624878 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-9ecd773f-177f-4df0-969a-dbb2ead8e80d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:25.633411 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-cf11b09b-5cb2-48f0-b086-3fdb2cae5f5c tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:25.633787 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9ecd773f-177f-4df0-969a-dbb2ead8e80d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.185551 Jul 11 14:41:25.634136 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-cf11b09b-5cb2-48f0-b086-3fdb2cae5f5c tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:25.634381 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 79/178] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:25 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:25.641957 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cf11b09b-5cb2-48f0-b086-3fdb2cae5f5c tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:25.642415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cf11b09b-5cb2-48f0-b086-3fdb2cae5f5c tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:25.643231 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cf11b09b-5cb2-48f0-b086-3fdb2cae5f5c tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:25.643726 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cf11b09b-5cb2-48f0-b086-3fdb2cae5f5c tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:25.801321 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-cf11b09b-5cb2-48f0-b086-3fdb2cae5f5c tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:25.811016 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-cf11b09b-5cb2-48f0-b086-3fdb2cae5f5c tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1552 microversion: 2.1 time: 0.184306 Jul 11 14:41:25.811593 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 100/179] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:25 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1552 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:26.650099 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c21c19b0-da1f-4f90-bb79-264719537ba1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:26.651381 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c21c19b0-da1f-4f90-bb79-264719537ba1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:26.657933 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c21c19b0-da1f-4f90-bb79-264719537ba1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:26.658390 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c21c19b0-da1f-4f90-bb79-264719537ba1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:26.659198 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c21c19b0-da1f-4f90-bb79-264719537ba1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:26.659640 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c21c19b0-da1f-4f90-bb79-264719537ba1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:26.831810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-b82eddf1-ba7a-44ee-8a79-129233908af5 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:26.832683 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-b82eddf1-ba7a-44ee-8a79-129233908af5 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:26.839920 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b82eddf1-ba7a-44ee-8a79-129233908af5 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:26.840895 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b82eddf1-ba7a-44ee-8a79-129233908af5 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:26.841654 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b82eddf1-ba7a-44ee-8a79-129233908af5 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:26.842215 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b82eddf1-ba7a-44ee-8a79-129233908af5 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:27.021693 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-b82eddf1-ba7a-44ee-8a79-129233908af5 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:27.022971 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c21c19b0-da1f-4f90-bb79-264719537ba1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:27.031098 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c21c19b0-da1f-4f90-bb79-264719537ba1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.384682 Jul 11 14:41:27.031622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 80/180] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:26 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:27.032988 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-b82eddf1-ba7a-44ee-8a79-129233908af5 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1552 microversion: 2.1 time: 0.207205 Jul 11 14:41:27.038012 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 101/181] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:26 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1552 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:28.055429 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a209e705-a11c-42b3-9b86-b13b4b89cbf0 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:28.056442 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a209e705-a11c-42b3-9b86-b13b4b89cbf0 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:28.060047 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-5e659e9e-3c3b-4543-8930-695184d5c023 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:28.061105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-5e659e9e-3c3b-4543-8930-695184d5c023 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:28.070113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5e659e9e-3c3b-4543-8930-695184d5c023 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:28.070782 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5e659e9e-3c3b-4543-8930-695184d5c023 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:28.071903 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5e659e9e-3c3b-4543-8930-695184d5c023 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:28.076236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a209e705-a11c-42b3-9b86-b13b4b89cbf0 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:28.076681 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a209e705-a11c-42b3-9b86-b13b4b89cbf0 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:28.077450 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a209e705-a11c-42b3-9b86-b13b4b89cbf0 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:28.077885 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a209e705-a11c-42b3-9b86-b13b4b89cbf0 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:28.082777 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5e659e9e-3c3b-4543-8930-695184d5c023 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:28.240322 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a209e705-a11c-42b3-9b86-b13b4b89cbf0 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:28.249037 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a209e705-a11c-42b3-9b86-b13b4b89cbf0 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1552 microversion: 2.1 time: 0.198801 Jul 11 14:41:28.249773 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 102/182] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:28 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1552 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:28.289085 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5e659e9e-3c3b-4543-8930-695184d5c023 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:28.293439 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:41:28.301498 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-5e659e9e-3c3b-4543-8930-695184d5c023 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.257083 Jul 11 14:41:28.312534 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 81/183] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:28 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:29.266527 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-96141643-8564-4fb3-9dc6-7fd67ce4ee18 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:29.267395 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-96141643-8564-4fb3-9dc6-7fd67ce4ee18 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:29.274013 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-96141643-8564-4fb3-9dc6-7fd67ce4ee18 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:29.274525 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-96141643-8564-4fb3-9dc6-7fd67ce4ee18 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:29.275402 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-96141643-8564-4fb3-9dc6-7fd67ce4ee18 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:29.275850 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-96141643-8564-4fb3-9dc6-7fd67ce4ee18 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:29.322029 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-565e9bbf-39c2-48eb-82c9-4f037c6deac5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:29.322829 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-565e9bbf-39c2-48eb-82c9-4f037c6deac5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:29.329838 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-565e9bbf-39c2-48eb-82c9-4f037c6deac5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:29.331355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-565e9bbf-39c2-48eb-82c9-4f037c6deac5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:29.332330 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-565e9bbf-39c2-48eb-82c9-4f037c6deac5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:29.332774 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-565e9bbf-39c2-48eb-82c9-4f037c6deac5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:29.430632 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-96141643-8564-4fb3-9dc6-7fd67ce4ee18 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:29.440230 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-96141643-8564-4fb3-9dc6-7fd67ce4ee18 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1552 microversion: 2.1 time: 0.177474 Jul 11 14:41:29.441439 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 103/184] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:29 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1552 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:29.494233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-565e9bbf-39c2-48eb-82c9-4f037c6deac5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:29.503758 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-565e9bbf-39c2-48eb-82c9-4f037c6deac5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.187390 Jul 11 14:41:29.504352 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 82/185] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:29 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:41:30.465208 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3f1e2187-179a-4bb6-a211-50562916060a tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:30.465934 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3f1e2187-179a-4bb6-a211-50562916060a tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:30.473235 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3f1e2187-179a-4bb6-a211-50562916060a tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:30.473693 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3f1e2187-179a-4bb6-a211-50562916060a tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:30.474619 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3f1e2187-179a-4bb6-a211-50562916060a tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:30.474940 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3f1e2187-179a-4bb6-a211-50562916060a tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:30.528066 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-5e82fd81-fc83-4c49-8287-eaf34e160a18 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:30.528066 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-5e82fd81-fc83-4c49-8287-eaf34e160a18 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:30.534491 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5e82fd81-fc83-4c49-8287-eaf34e160a18 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:30.534491 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5e82fd81-fc83-4c49-8287-eaf34e160a18 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:30.535200 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5e82fd81-fc83-4c49-8287-eaf34e160a18 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:30.535681 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5e82fd81-fc83-4c49-8287-eaf34e160a18 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:30.664895 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3f1e2187-179a-4bb6-a211-50562916060a tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:30.687908 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3f1e2187-179a-4bb6-a211-50562916060a tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1552 microversion: 2.1 time: 0.227165 Jul 11 14:41:30.688955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 104/186] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:30 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1552 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:30.813217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5e82fd81-fc83-4c49-8287-eaf34e160a18 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:30.821933 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-5e82fd81-fc83-4c49-8287-eaf34e160a18 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.300435 Jul 11 14:41:30.822914 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 83/187] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:30 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:30.961306 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-71cf3344-1e61-4e89-a877-e0b4f1e967a2 req-7c53c9c6-5626-4d2e-9085-e71b268e6f2a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a3e173ea-1d52-48e5-a5d1-89195af310c7", "name": "network-vif-deleted", "tag": "5f8fa020-0a9e-499b-a83e-8e2d443ee007"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:41:30.970461 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-71cf3344-1e61-4e89-a877-e0b4f1e967a2 req-7c53c9c6-5626-4d2e-9085-e71b268e6f2a service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:30.970953 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-71cf3344-1e61-4e89-a877-e0b4f1e967a2 req-7c53c9c6-5626-4d2e-9085-e71b268e6f2a service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:30.983338 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-71cf3344-1e61-4e89-a877-e0b4f1e967a2 req-7c53c9c6-5626-4d2e-9085-e71b268e6f2a service nova] Creating event network-vif-deleted:5f8fa020-0a9e-499b-a83e-8e2d443ee007 for instance a3e173ea-1d52-48e5-a5d1-89195af310c7 on hv-cinder-maste Jul 11 14:41:30.989533 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-71cf3344-1e61-4e89-a877-e0b4f1e967a2 req-7c53c9c6-5626-4d2e-9085-e71b268e6f2a service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031736 Jul 11 14:41:30.990240 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 105/188] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:41:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:41:31.706180 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-452abdf1-450a-4be1-b74a-49d382d8f763 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:31.707261 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-452abdf1-450a-4be1-b74a-49d382d8f763 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:31.716648 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-452abdf1-450a-4be1-b74a-49d382d8f763 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:31.717155 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-452abdf1-450a-4be1-b74a-49d382d8f763 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:31.717927 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-452abdf1-450a-4be1-b74a-49d382d8f763 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:31.718360 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-452abdf1-450a-4be1-b74a-49d382d8f763 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:31.749623 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-452abdf1-450a-4be1-b74a-49d382d8f763 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:31.750066 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-452abdf1-450a-4be1-b74a-49d382d8f763 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:31.750542 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.objects.instance [None req-452abdf1-450a-4be1-b74a-49d382d8f763 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lazy-loading 'fault' on Instance uuid a3e173ea-1d52-48e5-a5d1-89195af310c7 {{(pid=130160) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:41:31.810508 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-452abdf1-450a-4be1-b74a-49d382d8f763 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '85705ea8eb6b43e18d14eb5efefab96f', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'fd0be78cdcfc4195a0ced5868c14cda5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:31.823622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-452abdf1-450a-4be1-b74a-49d382d8f763 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 200 len: 1351 microversion: 2.1 time: 0.121818 Jul 11 14:41:31.834615 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 84/189] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:31 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 1351 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:31.841617 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-afafdd61-a93e-4ba5-88b4-a38fc4a82e89 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:31.843947 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-afafdd61-a93e-4ba5-88b4-a38fc4a82e89 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:31.858299 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-afafdd61-a93e-4ba5-88b4-a38fc4a82e89 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:31.859580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-afafdd61-a93e-4ba5-88b4-a38fc4a82e89 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:31.860819 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-afafdd61-a93e-4ba5-88b4-a38fc4a82e89 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:31.861408 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-afafdd61-a93e-4ba5-88b4-a38fc4a82e89 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:32.027843 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-afafdd61-a93e-4ba5-88b4-a38fc4a82e89 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:32.035366 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-afafdd61-a93e-4ba5-88b4-a38fc4a82e89 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.199269 Jul 11 14:41:32.035995 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 106/190] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:31 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:41:32.842113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d8b37605-339d-48b0-ab71-ea9a0f11f4c4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:32.842861 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d8b37605-339d-48b0-ab71-ea9a0f11f4c4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:32.849503 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d8b37605-339d-48b0-ab71-ea9a0f11f4c4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:32.849945 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d8b37605-339d-48b0-ab71-ea9a0f11f4c4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:32.850734 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d8b37605-339d-48b0-ab71-ea9a0f11f4c4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:32.851170 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d8b37605-339d-48b0-ab71-ea9a0f11f4c4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:32.865555 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-d8b37605-339d-48b0-ab71-ea9a0f11f4c4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] HTTP exception thrown: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d8b37605-339d-48b0-ab71-ea9a0f11f4c4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Request method failure captured: Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 HTTP/1.1 Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDZgp-PDUkBmvkGb-WZ8lfLW6OOGKQi64rMNkhclBPaMotG-hr8wtfsMWnGnkYsuRRWizjCWHCWeM_MHNhnGbjfvRJ7yhKJ0Wr6MK897SSpQJfE73n3kIuGsSAX1WvY9PMP4SMjyFjQJY_Odw0CDr0n4AYqg5yhxMNNNSQmO9i0buus Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: fd0be78cdcfc4195a0ced5868c14cda5 Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-DeleteServersTestJSON-245971247 Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:41:32.905485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:41:32.907023 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/fd0be78cdcfc4195a0ced5868c14cda5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/fd0be78cdcfc4195a0ced5868c14cda5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_fd0be78cdcfc4195a0ced5868c14cda5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/fd0be78cdcfc4195a0ced5868c14cda5"}]}] Jul 11 14:41:32.907023 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-DeleteServersTestJSON-245971247 Jul 11 14:41:32.907023 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: fd0be78cdcfc4195a0ced5868c14cda5 Jul 11 14:41:32.907023 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-DeleteServersTestJSON-245971247 Jul 11 14:41:32.907023 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-DeleteServersTestJSON-245971247-project Jul 11 14:41:32.907023 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:41:32.907023 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:41:32.907023 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 85705ea8eb6b43e18d14eb5efefab96f Jul 11 14:41:32.907023 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-DeleteServersTestJSON-245971247-project Jul 11 14:41:32.907023 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:41:32.907023 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:32.907023 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': 'a3e173ea-1d52-48e5-a5d1-89195af310c7'} Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:41:32.907972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:41:32.909377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:41:32.909377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:41:32.909377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:41:32.909377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:41:32.909377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:41:32.909377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:41:32.909377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:41:32.909377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:32.909377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:41:32.909377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:41:32.909377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:41:32.909377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:41:32.909377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:41:32.909377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:41:32.909377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:41:32.909377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:41:32.909377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:32.909377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:41:32.911032 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d8b37605-339d-48b0-ab71-ea9a0f11f4c4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Returning 404 to user: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:41:32.911563 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d8b37605-339d-48b0-ab71-ea9a0f11f4c4 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 404 len: 111 microversion: 2.1 time: 0.073815 Jul 11 14:41:32.912215 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 85/191] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:32 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 111 bytes in 75 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:41:33.016985 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e59da7d0-2184-41a8-b071-3c1ba18e441f tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:33.018484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e59da7d0-2184-41a8-b071-3c1ba18e441f tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:33.025821 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e59da7d0-2184-41a8-b071-3c1ba18e441f tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:33.026552 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e59da7d0-2184-41a8-b071-3c1ba18e441f tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:33.027481 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e59da7d0-2184-41a8-b071-3c1ba18e441f tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:33.028198 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e59da7d0-2184-41a8-b071-3c1ba18e441f tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:33.042912 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-e59da7d0-2184-41a8-b071-3c1ba18e441f tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] HTTP exception thrown: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:33.053209 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-5ec028f5-b185-400a-8129-1d525f557a66 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:33.054683 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-5ec028f5-b185-400a-8129-1d525f557a66 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e59da7d0-2184-41a8-b071-3c1ba18e441f tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Request method failure captured: Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7/os-volume_attachments HTTP/1.1 Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZgp-PDUkBmvkGb-WZ8lfLW6OOGKQi64rMNkhclBPaMotG-hr8wtfsMWnGnkYsuRRWizjCWHCWeM_MHNhnGbjfvRJ7yhKJ0Wr6MK897SSpQJfE73n3kIuGsSAX1WvY9PMP4SMjyFjQJY_Odw0CDr0n4AYqg5yhxMNNNSQmO9i0buus Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: fd0be78cdcfc4195a0ced5868c14cda5 Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-DeleteServersTestJSON-245971247 Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:41:33.061899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:41:33.063529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/fd0be78cdcfc4195a0ced5868c14cda5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/fd0be78cdcfc4195a0ced5868c14cda5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_fd0be78cdcfc4195a0ced5868c14cda5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/fd0be78cdcfc4195a0ced5868c14cda5"}]}] Jul 11 14:41:33.063529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-DeleteServersTestJSON-245971247 Jul 11 14:41:33.063529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: fd0be78cdcfc4195a0ced5868c14cda5 Jul 11 14:41:33.063529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-DeleteServersTestJSON-245971247 Jul 11 14:41:33.063529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-DeleteServersTestJSON-245971247-project Jul 11 14:41:33.063529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:41:33.063529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:41:33.063529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 85705ea8eb6b43e18d14eb5efefab96f Jul 11 14:41:33.063529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-DeleteServersTestJSON-245971247-project Jul 11 14:41:33.063529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:41:33.063529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:33.063529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'server_id': 'a3e173ea-1d52-48e5-a5d1-89195af310c7'} Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:41:33.064375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:33.065714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:41:33.067041 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5ec028f5-b185-400a-8129-1d525f557a66 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:33.067041 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5ec028f5-b185-400a-8129-1d525f557a66 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:33.067041 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5ec028f5-b185-400a-8129-1d525f557a66 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:33.067041 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5ec028f5-b185-400a-8129-1d525f557a66 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:33.068265 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e59da7d0-2184-41a8-b071-3c1ba18e441f tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Returning 404 to user: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:41:33.068897 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e59da7d0-2184-41a8-b071-3c1ba18e441f tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.055989 Jul 11 14:41:33.069411 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 107/192] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:41:33 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7/os-volume_attachments => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:41:33.260126 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5ec028f5-b185-400a-8129-1d525f557a66 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:33.271229 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-5ec028f5-b185-400a-8129-1d525f557a66 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.223040 Jul 11 14:41:33.271783 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 86/193] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:33 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:41:34.290473 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-41d4f1fb-042f-4939-bf69-87d54cbfeace tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:34.291403 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-41d4f1fb-042f-4939-bf69-87d54cbfeace tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:34.301115 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-41d4f1fb-042f-4939-bf69-87d54cbfeace tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:34.301673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-41d4f1fb-042f-4939-bf69-87d54cbfeace tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:34.302644 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-41d4f1fb-042f-4939-bf69-87d54cbfeace tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:34.303177 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-41d4f1fb-042f-4939-bf69-87d54cbfeace tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:34.306931 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e5f8c87a-93f5-4231-899a-d589aa6ddc8d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:34.311995 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-e5f8c87a-93f5-4231-899a-d589aa6ddc8d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:34.314405 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e5f8c87a-93f5-4231-899a-d589aa6ddc8d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:34.314971 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e5f8c87a-93f5-4231-899a-d589aa6ddc8d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:34.315832 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e5f8c87a-93f5-4231-899a-d589aa6ddc8d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:34.316332 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e5f8c87a-93f5-4231-899a-d589aa6ddc8d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:34.334763 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-e5f8c87a-93f5-4231-899a-d589aa6ddc8d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] HTTP exception thrown: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e5f8c87a-93f5-4231-899a-d589aa6ddc8d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Request method failure captured: Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: DELETE /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 HTTP/1.1 Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDZgp-PDUkBmvkGb-WZ8lfLW6OOGKQi64rMNkhclBPaMotG-hr8wtfsMWnGnkYsuRRWizjCWHCWeM_MHNhnGbjfvRJ7yhKJ0Wr6MK897SSpQJfE73n3kIuGsSAX1WvY9PMP4SMjyFjQJY_Odw0CDr0n4AYqg5yhxMNNNSQmO9i0buus Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: fd0be78cdcfc4195a0ced5868c14cda5 Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-DeleteServersTestJSON-245971247 Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:41:34.336394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:41:34.337863 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/fd0be78cdcfc4195a0ced5868c14cda5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/fd0be78cdcfc4195a0ced5868c14cda5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_fd0be78cdcfc4195a0ced5868c14cda5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/fd0be78cdcfc4195a0ced5868c14cda5"}]}] Jul 11 14:41:34.337863 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-DeleteServersTestJSON-245971247 Jul 11 14:41:34.337863 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: fd0be78cdcfc4195a0ced5868c14cda5 Jul 11 14:41:34.337863 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-DeleteServersTestJSON-245971247 Jul 11 14:41:34.337863 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-DeleteServersTestJSON-245971247-project Jul 11 14:41:34.337863 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:41:34.337863 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:41:34.337863 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 85705ea8eb6b43e18d14eb5efefab96f Jul 11 14:41:34.337863 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-DeleteServersTestJSON-245971247-project Jul 11 14:41:34.337863 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:41:34.337863 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:34.337863 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': 'a3e173ea-1d52-48e5-a5d1-89195af310c7'} Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:41:34.338693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:34.340105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:41:34.341504 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e5f8c87a-93f5-4231-899a-d589aa6ddc8d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Returning 404 to user: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:41:34.341504 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e5f8c87a-93f5-4231-899a-d589aa6ddc8d tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "DELETE /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 404 len: 111 microversion: 2.1 time: 0.038989 Jul 11 14:41:34.348376 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 87/194] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:41:34 2022] DELETE /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 11 14:41:34.360660 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c385c532-c60d-4655-a2d3-2d5dd0aa8985 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:34.361277 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c385c532-c60d-4655-a2d3-2d5dd0aa8985 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] [instance: a3e173ea-1d52-48e5-a5d1-89195af310c7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:34.368172 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c385c532-c60d-4655-a2d3-2d5dd0aa8985 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:34.368172 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c385c532-c60d-4655-a2d3-2d5dd0aa8985 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:34.369233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c385c532-c60d-4655-a2d3-2d5dd0aa8985 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:34.369664 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c385c532-c60d-4655-a2d3-2d5dd0aa8985 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:34.405323 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-c385c532-c60d-4655-a2d3-2d5dd0aa8985 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] HTTP exception thrown: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c385c532-c60d-4655-a2d3-2d5dd0aa8985 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Request method failure captured: Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 HTTP/1.1 Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDZgp-PDUkBmvkGb-WZ8lfLW6OOGKQi64rMNkhclBPaMotG-hr8wtfsMWnGnkYsuRRWizjCWHCWeM_MHNhnGbjfvRJ7yhKJ0Wr6MK897SSpQJfE73n3kIuGsSAX1WvY9PMP4SMjyFjQJY_Odw0CDr0n4AYqg5yhxMNNNSQmO9i0buus Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: fd0be78cdcfc4195a0ced5868c14cda5 Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-DeleteServersTestJSON-245971247 Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:41:34.408795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:41:34.410497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/fd0be78cdcfc4195a0ced5868c14cda5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/fd0be78cdcfc4195a0ced5868c14cda5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_fd0be78cdcfc4195a0ced5868c14cda5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/fd0be78cdcfc4195a0ced5868c14cda5"}]}] Jul 11 14:41:34.410497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-DeleteServersTestJSON-245971247 Jul 11 14:41:34.410497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: fd0be78cdcfc4195a0ced5868c14cda5 Jul 11 14:41:34.410497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-DeleteServersTestJSON-245971247 Jul 11 14:41:34.410497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-DeleteServersTestJSON-245971247-project Jul 11 14:41:34.410497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:41:34.410497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:41:34.410497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 85705ea8eb6b43e18d14eb5efefab96f Jul 11 14:41:34.410497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-DeleteServersTestJSON-245971247-project Jul 11 14:41:34.410497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:41:34.410497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:34.410497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': 'a3e173ea-1d52-48e5-a5d1-89195af310c7'} Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:41:34.414562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:41:34.416336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:41:34.416336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:41:34.416336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:41:34.416336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:41:34.416336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:41:34.416336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:41:34.416336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:41:34.416336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:34.416336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:41:34.416336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:41:34.416336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:41:34.416336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:41:34.416336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:41:34.416336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:41:34.416336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:41:34.416336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:41:34.416336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. Jul 11 14:41:34.416336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:41:34.417684 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c385c532-c60d-4655-a2d3-2d5dd0aa8985 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] Returning 404 to user: Instance a3e173ea-1d52-48e5-a5d1-89195af310c7 could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:41:34.417684 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c385c532-c60d-4655-a2d3-2d5dd0aa8985 tempest-DeleteServersTestJSON-245971247 tempest-DeleteServersTestJSON-245971247-project] 10.222.0.72 "GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7" status: 404 len: 111 microversion: 2.1 time: 0.064320 Jul 11 14:41:34.417684 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 88/195] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:34 2022] GET /compute/v2.1/servers/a3e173ea-1d52-48e5-a5d1-89195af310c7 => generated 111 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 11 14:41:34.562601 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-41d4f1fb-042f-4939-bf69-87d54cbfeace tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:34.575635 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-41d4f1fb-042f-4939-bf69-87d54cbfeace tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.288575 Jul 11 14:41:34.576992 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 108/196] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:34 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:35.592687 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-301742ac-ee5e-48c9-8da1-226e1f2e3ec6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:35.593379 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-301742ac-ee5e-48c9-8da1-226e1f2e3ec6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:35.599752 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-301742ac-ee5e-48c9-8da1-226e1f2e3ec6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:35.600195 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-301742ac-ee5e-48c9-8da1-226e1f2e3ec6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:35.600948 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-301742ac-ee5e-48c9-8da1-226e1f2e3ec6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:35.601380 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-301742ac-ee5e-48c9-8da1-226e1f2e3ec6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:35.736786 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-301742ac-ee5e-48c9-8da1-226e1f2e3ec6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:35.746120 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-301742ac-ee5e-48c9-8da1-226e1f2e3ec6 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.157176 Jul 11 14:41:35.746822 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 89/197] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:35 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:36.765741 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-5bd3aa98-9583-413d-a24e-d537f448ba4e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:36.766377 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-5bd3aa98-9583-413d-a24e-d537f448ba4e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:36.775122 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5bd3aa98-9583-413d-a24e-d537f448ba4e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:36.776284 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5bd3aa98-9583-413d-a24e-d537f448ba4e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:36.777453 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5bd3aa98-9583-413d-a24e-d537f448ba4e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:36.778294 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5bd3aa98-9583-413d-a24e-d537f448ba4e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:36.987959 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-5bd3aa98-9583-413d-a24e-d537f448ba4e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:36.999447 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-5bd3aa98-9583-413d-a24e-d537f448ba4e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.237695 Jul 11 14:41:37.000034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 109/198] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:36 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:38.017514 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-044a18a2-2902-4104-ae18-33c396beb45f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:38.018202 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-044a18a2-2902-4104-ae18-33c396beb45f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:38.025658 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-044a18a2-2902-4104-ae18-33c396beb45f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:38.026677 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-044a18a2-2902-4104-ae18-33c396beb45f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:38.027743 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-044a18a2-2902-4104-ae18-33c396beb45f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:38.028203 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-044a18a2-2902-4104-ae18-33c396beb45f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:38.500685 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-044a18a2-2902-4104-ae18-33c396beb45f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:38.508641 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-044a18a2-2902-4104-ae18-33c396beb45f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.495660 Jul 11 14:41:38.509220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 90/199] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:38 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:39.527416 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e358e19d-0f61-46fe-b332-c71a5b1dc297 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:39.529817 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e358e19d-0f61-46fe-b332-c71a5b1dc297 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:39.536276 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e358e19d-0f61-46fe-b332-c71a5b1dc297 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:39.536875 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e358e19d-0f61-46fe-b332-c71a5b1dc297 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:39.537509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e358e19d-0f61-46fe-b332-c71a5b1dc297 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:39.537947 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e358e19d-0f61-46fe-b332-c71a5b1dc297 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:39.560744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:41:39.679901 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e358e19d-0f61-46fe-b332-c71a5b1dc297 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:39.688215 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e358e19d-0f61-46fe-b332-c71a5b1dc297 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.165486 Jul 11 14:41:39.688795 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 110/200] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:39 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:40.706045 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-bfc26687-ab72-4915-97bc-df6078c2791a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:40.706846 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-bfc26687-ab72-4915-97bc-df6078c2791a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:40.714415 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bfc26687-ab72-4915-97bc-df6078c2791a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:40.714896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bfc26687-ab72-4915-97bc-df6078c2791a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:40.715666 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bfc26687-ab72-4915-97bc-df6078c2791a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:40.716158 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bfc26687-ab72-4915-97bc-df6078c2791a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:40.883535 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-bfc26687-ab72-4915-97bc-df6078c2791a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:40.895943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-bfc26687-ab72-4915-97bc-df6078c2791a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.193204 Jul 11 14:41:40.901366 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 91/201] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:40 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:41.912298 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c51e18b3-278a-42ab-813f-97a6b62aa50b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:41.912961 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c51e18b3-278a-42ab-813f-97a6b62aa50b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:41.919887 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=58,nova_api:UPDATE=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:41:41.922463 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c51e18b3-278a-42ab-813f-97a6b62aa50b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:41.923186 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c51e18b3-278a-42ab-813f-97a6b62aa50b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:41.924022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c51e18b3-278a-42ab-813f-97a6b62aa50b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:41.924442 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c51e18b3-278a-42ab-813f-97a6b62aa50b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:42.084678 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c51e18b3-278a-42ab-813f-97a6b62aa50b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:42.102950 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c51e18b3-278a-42ab-813f-97a6b62aa50b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.192702 Jul 11 14:41:42.104192 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 111/202] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:41 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:43.122751 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f9435bf8-d825-411c-9526-b01b12d4c0f2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:43.123492 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f9435bf8-d825-411c-9526-b01b12d4c0f2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:43.125850 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=47 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:41:43.131569 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f9435bf8-d825-411c-9526-b01b12d4c0f2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:43.132007 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f9435bf8-d825-411c-9526-b01b12d4c0f2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:43.132790 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f9435bf8-d825-411c-9526-b01b12d4c0f2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:43.133274 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f9435bf8-d825-411c-9526-b01b12d4c0f2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:43.334176 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f9435bf8-d825-411c-9526-b01b12d4c0f2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:43.343127 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f9435bf8-d825-411c-9526-b01b12d4c0f2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.224354 Jul 11 14:41:43.343625 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 92/203] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:43 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:44.365128 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8067f216-5e8a-4d2f-ae67-3f596df18b44 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:44.365128 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8067f216-5e8a-4d2f-ae67-3f596df18b44 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:44.370773 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8067f216-5e8a-4d2f-ae67-3f596df18b44 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:44.371582 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8067f216-5e8a-4d2f-ae67-3f596df18b44 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:44.372963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8067f216-5e8a-4d2f-ae67-3f596df18b44 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:44.373789 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8067f216-5e8a-4d2f-ae67-3f596df18b44 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:44.526694 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8067f216-5e8a-4d2f-ae67-3f596df18b44 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:44.539627 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8067f216-5e8a-4d2f-ae67-3f596df18b44 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.183451 Jul 11 14:41:44.540566 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 112/204] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:44 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:45.557439 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-838c7dfc-a609-4c96-98b7-17c1b08121c1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:45.564040 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-838c7dfc-a609-4c96-98b7-17c1b08121c1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:45.569855 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-838c7dfc-a609-4c96-98b7-17c1b08121c1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:45.570323 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-838c7dfc-a609-4c96-98b7-17c1b08121c1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:45.571125 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-838c7dfc-a609-4c96-98b7-17c1b08121c1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:45.571632 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-838c7dfc-a609-4c96-98b7-17c1b08121c1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:45.812077 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-838c7dfc-a609-4c96-98b7-17c1b08121c1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:45.819813 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-838c7dfc-a609-4c96-98b7-17c1b08121c1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.266861 Jul 11 14:41:45.829748 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 93/205] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:45 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:46.838078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2c58987f-5d1b-4c98-a4b9-50f698a67fce tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:46.838868 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2c58987f-5d1b-4c98-a4b9-50f698a67fce tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:46.845290 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2c58987f-5d1b-4c98-a4b9-50f698a67fce tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:46.846137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2c58987f-5d1b-4c98-a4b9-50f698a67fce tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:46.848343 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2c58987f-5d1b-4c98-a4b9-50f698a67fce tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:46.849394 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2c58987f-5d1b-4c98-a4b9-50f698a67fce tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:47.132088 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-2c58987f-5d1b-4c98-a4b9-50f698a67fce tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:47.141446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2c58987f-5d1b-4c98-a4b9-50f698a67fce tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.307644 Jul 11 14:41:47.142824 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 113/206] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:46 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:41:48.159586 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-5ea5d78d-743f-4449-8eed-d523eb3f5ec3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:48.160204 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-5ea5d78d-743f-4449-8eed-d523eb3f5ec3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:48.167689 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5ea5d78d-743f-4449-8eed-d523eb3f5ec3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:48.168364 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5ea5d78d-743f-4449-8eed-d523eb3f5ec3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:48.169322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5ea5d78d-743f-4449-8eed-d523eb3f5ec3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:48.169915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5ea5d78d-743f-4449-8eed-d523eb3f5ec3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:48.380453 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5ea5d78d-743f-4449-8eed-d523eb3f5ec3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:48.389148 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-5ea5d78d-743f-4449-8eed-d523eb3f5ec3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.233308 Jul 11 14:41:48.391142 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 94/207] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:48 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:49.410348 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-70adc66e-39d1-4e88-b2ef-0fed0acccf69 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:49.411309 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-70adc66e-39d1-4e88-b2ef-0fed0acccf69 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:49.422828 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-70adc66e-39d1-4e88-b2ef-0fed0acccf69 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:49.423638 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-70adc66e-39d1-4e88-b2ef-0fed0acccf69 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:49.424623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-70adc66e-39d1-4e88-b2ef-0fed0acccf69 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:49.426155 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-70adc66e-39d1-4e88-b2ef-0fed0acccf69 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:49.774173 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-70adc66e-39d1-4e88-b2ef-0fed0acccf69 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:49.784485 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-70adc66e-39d1-4e88-b2ef-0fed0acccf69 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.379018 Jul 11 14:41:49.788220 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 114/208] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:49 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:50.804805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-fd773856-3ee5-4c19-b891-ecac665a3991 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:50.806953 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-fd773856-3ee5-4c19-b891-ecac665a3991 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:50.817246 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd773856-3ee5-4c19-b891-ecac665a3991 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:50.817874 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd773856-3ee5-4c19-b891-ecac665a3991 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:50.818583 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd773856-3ee5-4c19-b891-ecac665a3991 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:50.819146 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd773856-3ee5-4c19-b891-ecac665a3991 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:51.075397 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-fd773856-3ee5-4c19-b891-ecac665a3991 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:51.103752 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-fd773856-3ee5-4c19-b891-ecac665a3991 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.306174 Jul 11 14:41:51.104586 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 95/209] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:50 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:52.125081 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-5196c1f8-e29a-408c-bbf6-54c6c5e3e0cb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:52.125803 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-5196c1f8-e29a-408c-bbf6-54c6c5e3e0cb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:52.140429 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5196c1f8-e29a-408c-bbf6-54c6c5e3e0cb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:52.140938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5196c1f8-e29a-408c-bbf6-54c6c5e3e0cb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:52.144246 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5196c1f8-e29a-408c-bbf6-54c6c5e3e0cb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:52.147765 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5196c1f8-e29a-408c-bbf6-54c6c5e3e0cb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:52.484158 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-5196c1f8-e29a-408c-bbf6-54c6c5e3e0cb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:52.497709 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-5196c1f8-e29a-408c-bbf6-54c6c5e3e0cb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.379305 Jul 11 14:41:52.498609 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 115/210] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:52 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:53.523939 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-dc1935f7-d00d-4c3a-bb7e-db2d952f5d17 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:53.525184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-dc1935f7-d00d-4c3a-bb7e-db2d952f5d17 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:53.541394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dc1935f7-d00d-4c3a-bb7e-db2d952f5d17 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:53.543752 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dc1935f7-d00d-4c3a-bb7e-db2d952f5d17 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:53.544575 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dc1935f7-d00d-4c3a-bb7e-db2d952f5d17 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:53.545914 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dc1935f7-d00d-4c3a-bb7e-db2d952f5d17 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:53.821029 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-dc1935f7-d00d-4c3a-bb7e-db2d952f5d17 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:53.830202 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-dc1935f7-d00d-4c3a-bb7e-db2d952f5d17 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1521 microversion: 2.1 time: 0.310800 Jul 11 14:41:53.831088 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 96/211] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:53 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1521 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:54.852701 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-631f2bd2-ebcd-4d83-b042-5d3320aab517 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:54.855144 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-631f2bd2-ebcd-4d83-b042-5d3320aab517 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:54.868586 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-631f2bd2-ebcd-4d83-b042-5d3320aab517 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:54.869146 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-631f2bd2-ebcd-4d83-b042-5d3320aab517 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:54.870643 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-631f2bd2-ebcd-4d83-b042-5d3320aab517 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:54.871333 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-631f2bd2-ebcd-4d83-b042-5d3320aab517 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:55.322808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-631f2bd2-ebcd-4d83-b042-5d3320aab517 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:55.331969 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-631f2bd2-ebcd-4d83-b042-5d3320aab517 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1501 microversion: 2.1 time: 0.488348 Jul 11 14:41:55.337420 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 116/212] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:54 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1501 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:55.356205 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c77e89cc-9f08-4ecf-8b9a-6a8e3e4de9d3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:55.363711 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c77e89cc-9f08-4ecf-8b9a-6a8e3e4de9d3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:55.376660 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c77e89cc-9f08-4ecf-8b9a-6a8e3e4de9d3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:55.410323 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c77e89cc-9f08-4ecf-8b9a-6a8e3e4de9d3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.033s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:55.413697 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c77e89cc-9f08-4ecf-8b9a-6a8e3e4de9d3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:55.414172 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c77e89cc-9f08-4ecf-8b9a-6a8e3e4de9d3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:55.756202 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c77e89cc-9f08-4ecf-8b9a-6a8e3e4de9d3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:55.779004 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c77e89cc-9f08-4ecf-8b9a-6a8e3e4de9d3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1515 microversion: 2.1 time: 0.429798 Jul 11 14:41:55.792007 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 97/213] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:55 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1515 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:57.695755 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-08515e02-61c1-43f8-afff-d900bd57317e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:41:57.696965 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-08515e02-61c1-43f8-afff-d900bd57317e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:57.716062 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-08515e02-61c1-43f8-afff-d900bd57317e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:57.716625 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-08515e02-61c1-43f8-afff-d900bd57317e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:57.717956 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-08515e02-61c1-43f8-afff-d900bd57317e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:57.718661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-08515e02-61c1-43f8-afff-d900bd57317e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:57.766324 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.objects.instance [None req-08515e02-61c1-43f8-afff-d900bd57317e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lazy-loading 'flavor' on Instance uuid a0498735-9e0a-4f14-990d-a6d5076e742a {{(pid=130159) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:41:57.964649 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-08515e02-61c1-43f8-afff-d900bd57317e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "POST /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a/action" status: 200 len: 32 microversion: 2.1 time: 0.273766 Jul 11 14:41:57.968972 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 117/214] 10.222.0.72 () {60 vars in 1277 bytes} [Mon Jul 11 14:41:57 2022] POST /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a/action => generated 32 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 11 14:41:57.981934 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8d0d0064-2aeb-4230-a2a7-bc190a699d73 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:57.983184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-8d0d0064-2aeb-4230-a2a7-bc190a699d73 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:58.013419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d0d0064-2aeb-4230-a2a7-bc190a699d73 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:58.013874 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d0d0064-2aeb-4230-a2a7-bc190a699d73 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:58.014752 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d0d0064-2aeb-4230-a2a7-bc190a699d73 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:58.015364 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d0d0064-2aeb-4230-a2a7-bc190a699d73 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:58.335714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-32098b3b-f510-4bc6-8705-412591d53edd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Action: 'create', calling method: .version_select at 0x7f59304c5700>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-2098849656", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMHKc8ibHykmtHUV0iDGxdir/7rAWBWUKVrUymgG4MlY7UHU++Dw1lb8hOqFJau2k0Hcsram1T0afXrxajFEFtFDAbbLUvAwpVMUZGq5khrNocD6DH0e+bm0uBkxfDN1dg=="}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:41:58.362112 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-32098b3b-f510-4bc6-8705-412591d53edd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Getting quotas for project 3d541b7a72dc4b4494a42bda6fa75026. Resources: {'key_pairs'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:41:58.367380 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-32098b3b-f510-4bc6-8705-412591d53edd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Getting quotas for user 0f8e300cd96e446db7430044e69d997a and project 3d541b7a72dc4b4494a42bda6fa75026. Resources: {'key_pairs'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:41:58.414880 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-32098b3b-f510-4bc6-8705-412591d53edd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.286553 Jul 11 14:41:58.416476 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 118/215] 10.222.0.72 () {60 vars in 1158 bytes} [Mon Jul 11 14:41:58 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:41:58.416664 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8d0d0064-2aeb-4230-a2a7-bc190a699d73 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:41:58.451616 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8d0d0064-2aeb-4230-a2a7-bc190a699d73 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.479238 Jul 11 14:41:58.453558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 98/216] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:57 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 483 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:41:59.487582 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-38127bda-82e8-4ddd-90b3-39a595d91c81 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:41:59.488468 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-38127bda-82e8-4ddd-90b3-39a595d91c81 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:41:59.504226 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-38127bda-82e8-4ddd-90b3-39a595d91c81 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:59.508396 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-38127bda-82e8-4ddd-90b3-39a595d91c81 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:59.510249 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-38127bda-82e8-4ddd-90b3-39a595d91c81 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:41:59.510741 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-38127bda-82e8-4ddd-90b3-39a595d91c81 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:41:59.995176 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-38127bda-82e8-4ddd-90b3-39a595d91c81 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:00.031268 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-38127bda-82e8-4ddd-90b3-39a595d91c81 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.558261 Jul 11 14:42:00.031941 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 119/217] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:41:59 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:01.049099 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a40bab26-5d32-4c8d-b1b1-f5fc21d36a02 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:01.050413 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a40bab26-5d32-4c8d-b1b1-f5fc21d36a02 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:01.057652 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e7b5bc84-2658-40fa-93ee-d892d46f4e70 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1616184459", "imageRef": "a588fa24-23b5-4e53-87c6-7e600ee3a5ef", "flavorRef": "452", "networks": [{"uuid": "c0778f11-32ed-4b63-9a0c-1270df9f375b"}]}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:01.058538 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a40bab26-5d32-4c8d-b1b1-f5fc21d36a02 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:01.058983 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a40bab26-5d32-4c8d-b1b1-f5fc21d36a02 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:01.060166 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a40bab26-5d32-4c8d-b1b1-f5fc21d36a02 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:01.060805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a40bab26-5d32-4c8d-b1b1-f5fc21d36a02 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:01.065483 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=93 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:01.118539 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.network.neutron [None req-e7b5bc84-2658-40fa-93ee-d892d46f4e70 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] validate_networks() for [('c0778f11-32ed-4b63-9a0c-1270df9f375b', None, None, None, None, None)] {{(pid=130159) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:42:01.217399 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a40bab26-5d32-4c8d-b1b1-f5fc21d36a02 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:01.225467 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a40bab26-5d32-4c8d-b1b1-f5fc21d36a02 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.180256 Jul 11 14:42:01.226263 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 99/218] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:01 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:01.480160 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-e7b5bc84-2658-40fa-93ee-d892d46f4e70 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Flavor limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:42:01.480582 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-e7b5bc84-2658-40fa-93ee-d892d46f4e70 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Image limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:42:01.481056 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-e7b5bc84-2658-40fa-93ee-d892d46f4e70 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Flavor pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:42:01.481465 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-e7b5bc84-2658-40fa-93ee-d892d46f4e70 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Image pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:42:01.481875 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-e7b5bc84-2658-40fa-93ee-d892d46f4e70 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:42:01.494393 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e7b5bc84-2658-40fa-93ee-d892d46f4e70 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:01.494946 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e7b5bc84-2658-40fa-93ee-d892d46f4e70 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:01.495811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e7b5bc84-2658-40fa-93ee-d892d46f4e70 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:01.496257 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e7b5bc84-2658-40fa-93ee-d892d46f4e70 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:01.514083 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-e7b5bc84-2658-40fa-93ee-d892d46f4e70 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Getting quotas for project 8e0ccc1e4683478889d90814a1305507. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:42:01.519457 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-e7b5bc84-2658-40fa-93ee-d892d46f4e70 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Getting quotas for user b7457a38a0594e8da4c053a5c6a4aa15 and project 8e0ccc1e4683478889d90814a1305507. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:42:01.537259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e7b5bc84-2658-40fa-93ee-d892d46f4e70 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Going to run 1 instances... {{(pid=130159) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:42:01.541419 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e7b5bc84-2658-40fa-93ee-d892d46f4e70 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a588fa24-23b5-4e53-87c6-7e600ee3a5ef',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130159) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:42:01.601204 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e7b5bc84-2658-40fa-93ee-d892d46f4e70 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.547913 Jul 11 14:42:01.602153 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 120/219] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:42:01 2022] POST /compute/v2.1/servers => generated 380 bytes in 549 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 11 14:42:01.612634 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-467a2908-d6af-4f05-8016-ccd058196841 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:01.616211 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-467a2908-d6af-4f05-8016-ccd058196841 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:01.691045 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-467a2908-d6af-4f05-8016-ccd058196841 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:01.698545 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-467a2908-d6af-4f05-8016-ccd058196841 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c" status: 200 len: 1219 microversion: 2.1 time: 0.089782 Jul 11 14:42:01.699670 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 100/220] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:01 2022] GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:02.244815 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9ec6f65b-0faf-4c6a-a937-6db02cd1ab28 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:02.245455 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9ec6f65b-0faf-4c6a-a937-6db02cd1ab28 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:02.256213 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9ec6f65b-0faf-4c6a-a937-6db02cd1ab28 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:02.256928 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9ec6f65b-0faf-4c6a-a937-6db02cd1ab28 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:02.257774 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9ec6f65b-0faf-4c6a-a937-6db02cd1ab28 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:02.259060 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9ec6f65b-0faf-4c6a-a937-6db02cd1ab28 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:02.450656 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9ec6f65b-0faf-4c6a-a937-6db02cd1ab28 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:02.458461 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9ec6f65b-0faf-4c6a-a937-6db02cd1ab28 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.219616 Jul 11 14:42:02.459041 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 121/221] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:02 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:02.718413 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-fd79242a-2d9f-4563-baea-e667135ca01c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:02.719257 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-fd79242a-2d9f-4563-baea-e667135ca01c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:02.730274 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd79242a-2d9f-4563-baea-e667135ca01c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:02.731024 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd79242a-2d9f-4563-baea-e667135ca01c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:02.731874 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd79242a-2d9f-4563-baea-e667135ca01c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:02.733024 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd79242a-2d9f-4563-baea-e667135ca01c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:02.805672 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-fd79242a-2d9f-4563-baea-e667135ca01c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:02.814062 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-fd79242a-2d9f-4563-baea-e667135ca01c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c" status: 200 len: 1271 microversion: 2.1 time: 0.101048 Jul 11 14:42:02.815198 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 101/222] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:02 2022] GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c => generated 1271 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:03.481611 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-37b32c6c-b398-4946-9daf-25c0802e8022 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:03.482259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-37b32c6c-b398-4946-9daf-25c0802e8022 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:03.495452 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-37b32c6c-b398-4946-9daf-25c0802e8022 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:03.495920 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-37b32c6c-b398-4946-9daf-25c0802e8022 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:03.496731 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-37b32c6c-b398-4946-9daf-25c0802e8022 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:03.497183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-37b32c6c-b398-4946-9daf-25c0802e8022 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:03.821927 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-37b32c6c-b398-4946-9daf-25c0802e8022 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:03.832600 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-6571b501-59f6-4e00-b716-120171f556eb tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:03.834334 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-37b32c6c-b398-4946-9daf-25c0802e8022 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.358979 Jul 11 14:42:03.835674 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-6571b501-59f6-4e00-b716-120171f556eb tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:03.837966 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 122/223] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:03 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:03.846864 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6571b501-59f6-4e00-b716-120171f556eb tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:03.847577 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6571b501-59f6-4e00-b716-120171f556eb tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:03.848536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6571b501-59f6-4e00-b716-120171f556eb tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:03.849129 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6571b501-59f6-4e00-b716-120171f556eb tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:04.429095 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6571b501-59f6-4e00-b716-120171f556eb tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:04.440856 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-6571b501-59f6-4e00-b716-120171f556eb tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c" status: 200 len: 1319 microversion: 2.1 time: 0.611563 Jul 11 14:42:04.442535 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 102/224] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:03 2022] GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c => generated 1319 bytes in 615 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:04.853283 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e30a701f-fc5c-4220-8f00-53fe1d32a849 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:04.853933 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e30a701f-fc5c-4220-8f00-53fe1d32a849 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:04.861104 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e30a701f-fc5c-4220-8f00-53fe1d32a849 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:04.861786 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e30a701f-fc5c-4220-8f00-53fe1d32a849 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:04.862796 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e30a701f-fc5c-4220-8f00-53fe1d32a849 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:04.863241 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e30a701f-fc5c-4220-8f00-53fe1d32a849 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:05.027581 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e30a701f-fc5c-4220-8f00-53fe1d32a849 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:05.046457 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e30a701f-fc5c-4220-8f00-53fe1d32a849 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.197045 Jul 11 14:42:05.058989 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 123/225] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:04 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:05.085205 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-e8d9541e-061c-4fe4-8125-16f5d54c062e req-3f16e40b-ccd0-4375-a5cd-f21c577f4880 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "751f3bb7-cb4e-43c3-aab8-554e34dc828c", "tag": "90c2c8c8-bf06-4bf9-962e-25a332ea4dba"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:05.096241 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-e8d9541e-061c-4fe4-8125-16f5d54c062e req-3f16e40b-ccd0-4375-a5cd-f21c577f4880 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:05.096708 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-e8d9541e-061c-4fe4-8125-16f5d54c062e req-3f16e40b-ccd0-4375-a5cd-f21c577f4880 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:05.112821 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-e8d9541e-061c-4fe4-8125-16f5d54c062e req-3f16e40b-ccd0-4375-a5cd-f21c577f4880 service nova] Creating event network-changed:90c2c8c8-bf06-4bf9-962e-25a332ea4dba for instance 751f3bb7-cb4e-43c3-aab8-554e34dc828c on hv-cinder-maste Jul 11 14:42:05.119413 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-e8d9541e-061c-4fe4-8125-16f5d54c062e req-3f16e40b-ccd0-4375-a5cd-f21c577f4880 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038632 Jul 11 14:42:05.120620 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 103/226] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:42:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:42:05.459482 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e55b97f5-d8ec-44cf-bdb8-42d6730f7c20 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:05.460347 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e55b97f5-d8ec-44cf-bdb8-42d6730f7c20 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:05.467006 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e55b97f5-d8ec-44cf-bdb8-42d6730f7c20 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:05.467300 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e55b97f5-d8ec-44cf-bdb8-42d6730f7c20 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:05.468542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e55b97f5-d8ec-44cf-bdb8-42d6730f7c20 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:05.468977 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e55b97f5-d8ec-44cf-bdb8-42d6730f7c20 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:05.627959 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e55b97f5-d8ec-44cf-bdb8-42d6730f7c20 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:05.636855 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e55b97f5-d8ec-44cf-bdb8-42d6730f7c20 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c" status: 200 len: 1319 microversion: 2.1 time: 0.181550 Jul 11 14:42:05.645348 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 124/227] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:05 2022] GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c => generated 1319 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:06.066237 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-edeb0cc9-1311-4383-a27d-3b77455e3dae tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:06.066860 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-edeb0cc9-1311-4383-a27d-3b77455e3dae tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:06.073712 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-edeb0cc9-1311-4383-a27d-3b77455e3dae tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:06.074164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-edeb0cc9-1311-4383-a27d-3b77455e3dae tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:06.074958 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-edeb0cc9-1311-4383-a27d-3b77455e3dae tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:06.075431 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-edeb0cc9-1311-4383-a27d-3b77455e3dae tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:06.206613 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-edeb0cc9-1311-4383-a27d-3b77455e3dae tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:06.214622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-edeb0cc9-1311-4383-a27d-3b77455e3dae tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.152639 Jul 11 14:42:06.215184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 104/228] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:06 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:06.653637 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-ae918ccf-b832-41d3-923f-12ea8dfaf605 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:06.654224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-ae918ccf-b832-41d3-923f-12ea8dfaf605 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:06.660400 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ae918ccf-b832-41d3-923f-12ea8dfaf605 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:06.660845 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ae918ccf-b832-41d3-923f-12ea8dfaf605 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:06.661604 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ae918ccf-b832-41d3-923f-12ea8dfaf605 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:06.662030 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ae918ccf-b832-41d3-923f-12ea8dfaf605 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:06.809928 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-ae918ccf-b832-41d3-923f-12ea8dfaf605 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:06.817444 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-ae918ccf-b832-41d3-923f-12ea8dfaf605 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c" status: 200 len: 1319 microversion: 2.1 time: 0.168044 Jul 11 14:42:06.818027 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 125/229] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:06 2022] GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c => generated 1319 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:07.231966 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-1f5f17d8-e412-4fd2-a709-ae829bb7c01c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:07.232575 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-1f5f17d8-e412-4fd2-a709-ae829bb7c01c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:07.245569 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1f5f17d8-e412-4fd2-a709-ae829bb7c01c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:07.246332 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1f5f17d8-e412-4fd2-a709-ae829bb7c01c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:07.247170 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1f5f17d8-e412-4fd2-a709-ae829bb7c01c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:07.247774 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1f5f17d8-e412-4fd2-a709-ae829bb7c01c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:07.385031 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-1f5f17d8-e412-4fd2-a709-ae829bb7c01c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:07.393613 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-1f5f17d8-e412-4fd2-a709-ae829bb7c01c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.166002 Jul 11 14:42:07.413261 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 105/230] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:07 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:07.834667 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-220aed60-b1a9-49b8-98bf-4a72d74b59e4 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:07.835346 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-220aed60-b1a9-49b8-98bf-4a72d74b59e4 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:07.841373 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-220aed60-b1a9-49b8-98bf-4a72d74b59e4 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:07.841820 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-220aed60-b1a9-49b8-98bf-4a72d74b59e4 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:07.842621 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-220aed60-b1a9-49b8-98bf-4a72d74b59e4 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:07.843053 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-220aed60-b1a9-49b8-98bf-4a72d74b59e4 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:08.216942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-220aed60-b1a9-49b8-98bf-4a72d74b59e4 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:08.220967 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:08.227562 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-220aed60-b1a9-49b8-98bf-4a72d74b59e4 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c" status: 200 len: 1319 microversion: 2.1 time: 0.396615 Jul 11 14:42:08.228136 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 126/231] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:07 2022] GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c => generated 1319 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:08.411836 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-38bad322-22ee-400e-b6ad-b3d7505a2316 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:08.413174 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-38bad322-22ee-400e-b6ad-b3d7505a2316 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:08.422902 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-38bad322-22ee-400e-b6ad-b3d7505a2316 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:08.423366 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-38bad322-22ee-400e-b6ad-b3d7505a2316 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:08.424151 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-38bad322-22ee-400e-b6ad-b3d7505a2316 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:08.424595 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-38bad322-22ee-400e-b6ad-b3d7505a2316 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:08.568269 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-38bad322-22ee-400e-b6ad-b3d7505a2316 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:08.576975 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-38bad322-22ee-400e-b6ad-b3d7505a2316 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.168705 Jul 11 14:42:08.577502 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 106/232] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:08 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:09.245338 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3320b8ab-9492-4fbc-8467-b9fa25257673 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:09.245972 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3320b8ab-9492-4fbc-8467-b9fa25257673 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:09.253211 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3320b8ab-9492-4fbc-8467-b9fa25257673 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:09.253603 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3320b8ab-9492-4fbc-8467-b9fa25257673 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:09.254376 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3320b8ab-9492-4fbc-8467-b9fa25257673 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:09.254862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3320b8ab-9492-4fbc-8467-b9fa25257673 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:09.390079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3320b8ab-9492-4fbc-8467-b9fa25257673 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:09.397839 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3320b8ab-9492-4fbc-8467-b9fa25257673 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c" status: 200 len: 1319 microversion: 2.1 time: 0.157233 Jul 11 14:42:09.398550 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 127/233] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:09 2022] GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:09.594975 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-661318e7-a0df-42f1-a06e-c856a1d050cc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:09.595589 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-661318e7-a0df-42f1-a06e-c856a1d050cc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:09.602864 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-661318e7-a0df-42f1-a06e-c856a1d050cc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:09.603315 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-661318e7-a0df-42f1-a06e-c856a1d050cc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:09.604345 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-661318e7-a0df-42f1-a06e-c856a1d050cc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:09.605478 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-661318e7-a0df-42f1-a06e-c856a1d050cc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:09.644822 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-1de16180-bb5b-4587-944d-5b50ddb86705 req-badf3d83-f3a6-4347-b01e-3fb35928b174 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "751f3bb7-cb4e-43c3-aab8-554e34dc828c", "name": "network-vif-plugged", "status": "completed", "tag": "90c2c8c8-bf06-4bf9-962e-25a332ea4dba"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:09.673807 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-1de16180-bb5b-4587-944d-5b50ddb86705 req-badf3d83-f3a6-4347-b01e-3fb35928b174 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:09.676077 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-1de16180-bb5b-4587-944d-5b50ddb86705 req-badf3d83-f3a6-4347-b01e-3fb35928b174 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:09.713843 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-1de16180-bb5b-4587-944d-5b50ddb86705 req-badf3d83-f3a6-4347-b01e-3fb35928b174 service nova] Creating event network-vif-plugged:90c2c8c8-bf06-4bf9-962e-25a332ea4dba for instance 751f3bb7-cb4e-43c3-aab8-554e34dc828c on hv-cinder-maste Jul 11 14:42:09.733616 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-1de16180-bb5b-4587-944d-5b50ddb86705 req-badf3d83-f3a6-4347-b01e-3fb35928b174 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091749 Jul 11 14:42:09.734475 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 128/234] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:42:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:42:09.749114 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-b2ea464c-02df-4697-9737-b520df36b088 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:09.749819 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-b2ea464c-02df-4697-9737-b520df36b088 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:09.753109 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-661318e7-a0df-42f1-a06e-c856a1d050cc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:09.758356 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b2ea464c-02df-4697-9737-b520df36b088 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:09.759549 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b2ea464c-02df-4697-9737-b520df36b088 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:09.760723 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b2ea464c-02df-4697-9737-b520df36b088 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:09.761515 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b2ea464c-02df-4697-9737-b520df36b088 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:09.762273 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-661318e7-a0df-42f1-a06e-c856a1d050cc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.170990 Jul 11 14:42:09.762913 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 107/235] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:09 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:09.795094 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-b2ea464c-02df-4697-9737-b520df36b088 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/ips" status: 200 len: 149 microversion: 2.1 time: 0.052528 Jul 11 14:42:09.795630 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 129/236] 10.222.0.72 () {58 vars in 1247 bytes} [Mon Jul 11 14:42:09 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/ips => generated 149 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:42:10.417818 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9b5690d5-632f-44b4-b9d0-513c405c02e3 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:10.418447 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-9b5690d5-632f-44b4-b9d0-513c405c02e3 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:10.424773 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9b5690d5-632f-44b4-b9d0-513c405c02e3 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:10.425222 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9b5690d5-632f-44b4-b9d0-513c405c02e3 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:10.425992 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9b5690d5-632f-44b4-b9d0-513c405c02e3 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:10.426418 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9b5690d5-632f-44b4-b9d0-513c405c02e3 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:10.567777 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-9b5690d5-632f-44b4-b9d0-513c405c02e3 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:10.575826 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9b5690d5-632f-44b4-b9d0-513c405c02e3 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c" status: 200 len: 1319 microversion: 2.1 time: 0.162236 Jul 11 14:42:10.576407 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 108/237] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:10 2022] GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c => generated 1319 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:10.779885 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-6a2c33e9-a48c-4b46-96e3-d8bf8dec57c8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:10.780489 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-6a2c33e9-a48c-4b46-96e3-d8bf8dec57c8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:10.786717 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6a2c33e9-a48c-4b46-96e3-d8bf8dec57c8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:10.787171 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6a2c33e9-a48c-4b46-96e3-d8bf8dec57c8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:10.787935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6a2c33e9-a48c-4b46-96e3-d8bf8dec57c8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:10.788378 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6a2c33e9-a48c-4b46-96e3-d8bf8dec57c8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:10.926288 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-6a2c33e9-a48c-4b46-96e3-d8bf8dec57c8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:10.934645 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-6a2c33e9-a48c-4b46-96e3-d8bf8dec57c8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.158672 Jul 11 14:42:10.935213 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 130/238] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:10 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:11.186657 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-1d032d83-3eac-4591-9463-e5d217170e94 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "69ca4bc6-3d1c-4bef-ae85-7e63c008f48d"}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:11.187750 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-1d032d83-3eac-4591-9463-e5d217170e94 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:11.194970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1d032d83-3eac-4591-9463-e5d217170e94 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:11.195371 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1d032d83-3eac-4591-9463-e5d217170e94 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:11.196675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1d032d83-3eac-4591-9463-e5d217170e94 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:11.197117 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1d032d83-3eac-4591-9463-e5d217170e94 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:11.593235 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9c3637c8-e3aa-42ac-ba7b-367c6ee1b812 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:11.593877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9c3637c8-e3aa-42ac-ba7b-367c6ee1b812 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:11.595919 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=2 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:11.596583 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=10 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:11.603926 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9c3637c8-e3aa-42ac-ba7b-367c6ee1b812 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:11.604470 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9c3637c8-e3aa-42ac-ba7b-367c6ee1b812 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:11.605250 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9c3637c8-e3aa-42ac-ba7b-367c6ee1b812 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:11.605687 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9c3637c8-e3aa-42ac-ba7b-367c6ee1b812 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:11.696659 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=2 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:11.744515 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9c3637c8-e3aa-42ac-ba7b-367c6ee1b812 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:11.751935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9c3637c8-e3aa-42ac-ba7b-367c6ee1b812 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c" status: 200 len: 1319 microversion: 2.1 time: 0.162567 Jul 11 14:42:11.752506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 131/239] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:11 2022] GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c => generated 1319 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:11.951780 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-30f45bed-2787-4409-b589-02db15b9c910 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:11.952409 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-30f45bed-2787-4409-b589-02db15b9c910 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:11.954198 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=62,nova_api:INSERT=4 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:11.960097 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-30f45bed-2787-4409-b589-02db15b9c910 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:11.960598 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-30f45bed-2787-4409-b589-02db15b9c910 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:11.961375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-30f45bed-2787-4409-b589-02db15b9c910 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:11.961820 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-30f45bed-2787-4409-b589-02db15b9c910 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:12.112748 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-30f45bed-2787-4409-b589-02db15b9c910 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:12.121206 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-30f45bed-2787-4409-b589-02db15b9c910 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.173328 Jul 11 14:42:12.121801 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 132/240] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:11 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:12.170386 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-1d032d83-3eac-4591-9463-e5d217170e94 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "POST /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.988518 Jul 11 14:42:12.171154 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 109/241] 10.222.0.72 () {60 vars in 1322 bytes} [Mon Jul 11 14:42:11 2022] POST /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments => generated 194 bytes in 990 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:42:12.759049 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a92f9ed6-ffad-4e7f-9956-89dbb9753ff6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:12.759540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a92f9ed6-ffad-4e7f-9956-89dbb9753ff6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:12.768143 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a92f9ed6-ffad-4e7f-9956-89dbb9753ff6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:12.768594 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a92f9ed6-ffad-4e7f-9956-89dbb9753ff6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:12.769363 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a92f9ed6-ffad-4e7f-9956-89dbb9753ff6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:12.769838 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a92f9ed6-ffad-4e7f-9956-89dbb9753ff6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:12.770279 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-57978d29-3c37-4a52-af1a-55fcf8b70b35 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:12.771102 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-57978d29-3c37-4a52-af1a-55fcf8b70b35 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:12.777512 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-57978d29-3c37-4a52-af1a-55fcf8b70b35 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:12.778071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-57978d29-3c37-4a52-af1a-55fcf8b70b35 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:12.778832 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-57978d29-3c37-4a52-af1a-55fcf8b70b35 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:12.779447 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-57978d29-3c37-4a52-af1a-55fcf8b70b35 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:12.795193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a92f9ed6-ffad-4e7f-9956-89dbb9753ff6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/ips" status: 200 len: 151 microversion: 2.20 time: 0.040017 Jul 11 14:42:12.795766 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 133/242] 10.222.0.72 () {60 vars in 1288 bytes} [Mon Jul 11 14:42:12 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/ips => generated 151 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:42:12.923502 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-57978d29-3c37-4a52-af1a-55fcf8b70b35 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:12.931069 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-57978d29-3c37-4a52-af1a-55fcf8b70b35 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c" status: 200 len: 1508 microversion: 2.1 time: 0.165624 Jul 11 14:42:12.931646 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 110/243] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:12 2022] GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c => generated 1508 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:13.139152 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-369807dd-683a-4787-9e9e-13c89474f9ac tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:13.139860 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-369807dd-683a-4787-9e9e-13c89474f9ac tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:13.146648 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-369807dd-683a-4787-9e9e-13c89474f9ac tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:13.147349 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-369807dd-683a-4787-9e9e-13c89474f9ac tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:13.148396 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-369807dd-683a-4787-9e9e-13c89474f9ac tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:13.149232 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-369807dd-683a-4787-9e9e-13c89474f9ac tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:13.307770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-369807dd-683a-4787-9e9e-13c89474f9ac tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:13.310358 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-369807dd-683a-4787-9e9e-13c89474f9ac tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.175265 Jul 11 14:42:13.310960 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 134/244] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:13 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:13.533915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-1876e1db-3134-4e7d-b946-f2ebeb9a1e2c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:13.534455 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-1876e1db-3134-4e7d-b946-f2ebeb9a1e2c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:13.537165 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=40 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:13.544128 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1876e1db-3134-4e7d-b946-f2ebeb9a1e2c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:13.544723 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1876e1db-3134-4e7d-b946-f2ebeb9a1e2c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:13.545337 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1876e1db-3134-4e7d-b946-f2ebeb9a1e2c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:13.545758 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1876e1db-3134-4e7d-b946-f2ebeb9a1e2c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:13.569825 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-1876e1db-3134-4e7d-b946-f2ebeb9a1e2c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d/ips" status: 200 len: 146 microversion: 2.1 time: 0.039787 Jul 11 14:42:13.575036 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 111/245] 10.222.0.72 () {58 vars in 1247 bytes} [Mon Jul 11 14:42:13 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d/ips => generated 146 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:42:14.327576 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-16b048ab-45ab-47f0-9e30-4228e8fe3aea tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:14.328188 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-16b048ab-45ab-47f0-9e30-4228e8fe3aea tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:14.334678 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-16b048ab-45ab-47f0-9e30-4228e8fe3aea tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:14.335367 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-16b048ab-45ab-47f0-9e30-4228e8fe3aea tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:14.336046 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-16b048ab-45ab-47f0-9e30-4228e8fe3aea tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:14.337156 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-16b048ab-45ab-47f0-9e30-4228e8fe3aea tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:14.468631 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-16b048ab-45ab-47f0-9e30-4228e8fe3aea tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:14.476079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-16b048ab-45ab-47f0-9e30-4228e8fe3aea tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.152376 Jul 11 14:42:14.476649 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 135/246] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:14 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:15.493315 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-ed7f9a2f-d458-4ebf-8f57-053e0fde6bd2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:15.493922 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-ed7f9a2f-d458-4ebf-8f57-053e0fde6bd2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:15.501056 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ed7f9a2f-d458-4ebf-8f57-053e0fde6bd2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:15.501449 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ed7f9a2f-d458-4ebf-8f57-053e0fde6bd2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:15.502315 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ed7f9a2f-d458-4ebf-8f57-053e0fde6bd2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:15.502871 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ed7f9a2f-d458-4ebf-8f57-053e0fde6bd2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:15.634142 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-ed7f9a2f-d458-4ebf-8f57-053e0fde6bd2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:15.641734 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-ed7f9a2f-d458-4ebf-8f57-053e0fde6bd2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.152545 Jul 11 14:42:15.642154 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 112/247] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:15 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:16.660275 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-78a97f7d-ebd2-45f9-b976-d1fd729becb5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:16.666825 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-78a97f7d-ebd2-45f9-b976-d1fd729becb5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:16.681739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-78a97f7d-ebd2-45f9-b976-d1fd729becb5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:16.682195 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-78a97f7d-ebd2-45f9-b976-d1fd729becb5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:16.683007 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-78a97f7d-ebd2-45f9-b976-d1fd729becb5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:16.683442 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-78a97f7d-ebd2-45f9-b976-d1fd729becb5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:16.864500 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-78a97f7d-ebd2-45f9-b976-d1fd729becb5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:16.876836 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-78a97f7d-ebd2-45f9-b976-d1fd729becb5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.223029 Jul 11 14:42:16.878415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 136/248] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:16 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:17.862051 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-fd346eb7-9bac-40d0-8e43-d584874fcbdd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "05e12cf2-ae62-436e-b69f-67ab88493cb5"}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:17.863238 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-fd346eb7-9bac-40d0-8e43-d584874fcbdd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:17.869795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd346eb7-9bac-40d0-8e43-d584874fcbdd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:17.870247 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd346eb7-9bac-40d0-8e43-d584874fcbdd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:17.871052 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd346eb7-9bac-40d0-8e43-d584874fcbdd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:17.871492 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fd346eb7-9bac-40d0-8e43-d584874fcbdd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:17.894190 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-849511ca-cd0d-42ad-a136-1edac8cf1f5b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:17.894837 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-849511ca-cd0d-42ad-a136-1edac8cf1f5b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:17.901357 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-849511ca-cd0d-42ad-a136-1edac8cf1f5b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:17.901804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-849511ca-cd0d-42ad-a136-1edac8cf1f5b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:17.902595 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-849511ca-cd0d-42ad-a136-1edac8cf1f5b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:17.903029 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-849511ca-cd0d-42ad-a136-1edac8cf1f5b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:18.034187 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-849511ca-cd0d-42ad-a136-1edac8cf1f5b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:18.042537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-849511ca-cd0d-42ad-a136-1edac8cf1f5b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.152160 Jul 11 14:42:18.043118 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 137/249] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:17 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:18.560893 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a7a97d50-b698-4bc8-b4b6-1efd1f0a9cca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:18.561429 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a7a97d50-b698-4bc8-b4b6-1efd1f0a9cca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:18.567905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a7a97d50-b698-4bc8-b4b6-1efd1f0a9cca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:18.568591 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a7a97d50-b698-4bc8-b4b6-1efd1f0a9cca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:18.569339 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a7a97d50-b698-4bc8-b4b6-1efd1f0a9cca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:18.570129 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a7a97d50-b698-4bc8-b4b6-1efd1f0a9cca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:18.599163 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-fd346eb7-9bac-40d0-8e43-d584874fcbdd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "POST /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.738456 Jul 11 14:42:18.599872 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 113/250] 10.222.0.72 () {60 vars in 1322 bytes} [Mon Jul 11 14:42:17 2022] POST /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d/os-volume_attachments => generated 194 bytes in 743 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:42:18.608504 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.objects.instance [None req-a7a97d50-b698-4bc8-b4b6-1efd1f0a9cca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lazy-loading 'flavor' on Instance uuid 13da005a-e085-43aa-8a18-1fe85e4e95af {{(pid=130159) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:42:18.911939 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a7a97d50-b698-4bc8-b4b6-1efd1f0a9cca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "POST /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/action" status: 202 len: 0 microversion: 2.20 time: 0.355237 Jul 11 14:42:18.913041 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 138/251] 10.222.0.72 () {62 vars in 1318 bytes} [Mon Jul 11 14:42:18 2022] POST /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/action => generated 0 bytes in 357 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Jul 11 14:42:18.923641 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-00f4c951-fb4a-41db-a268-903990e67e35 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:18.924465 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-00f4c951-fb4a-41db-a268-903990e67e35 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:18.931731 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-00f4c951-fb4a-41db-a268-903990e67e35 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:18.932216 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-00f4c951-fb4a-41db-a268-903990e67e35 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:18.932996 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-00f4c951-fb4a-41db-a268-903990e67e35 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:18.933534 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-00f4c951-fb4a-41db-a268-903990e67e35 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:19.062714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-bbddb2b2-06fd-451e-a8f0-60f9a5cb8dab tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:19.066806 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-bbddb2b2-06fd-451e-a8f0-60f9a5cb8dab tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:19.074643 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bbddb2b2-06fd-451e-a8f0-60f9a5cb8dab tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:19.075877 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-00f4c951-fb4a-41db-a268-903990e67e35 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:19.076018 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bbddb2b2-06fd-451e-a8f0-60f9a5cb8dab tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:19.076697 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bbddb2b2-06fd-451e-a8f0-60f9a5cb8dab tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:19.077135 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bbddb2b2-06fd-451e-a8f0-60f9a5cb8dab tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:19.085752 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-00f4c951-fb4a-41db-a268-903990e67e35 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:19.087755 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-00f4c951-fb4a-41db-a268-903990e67e35 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:19.089188 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-00f4c951-fb4a-41db-a268-903990e67e35 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.170593 Jul 11 14:42:19.101850 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 114/252] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:18 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:42:19.223249 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-bbddb2b2-06fd-451e-a8f0-60f9a5cb8dab tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:19.231317 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-bbddb2b2-06fd-451e-a8f0-60f9a5cb8dab tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.174146 Jul 11 14:42:19.231882 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 139/253] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:19 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:19.785980 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9c812db1-1115-4190-b531-376c819c5115 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Action: 'create', calling method: .version_select at 0x7f5930240af0>, body: {"keypair": {"name": "tempest-keypair-1046205418", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMrS+rumKJmAaH9ddavs2FK6YbUkmz31EF9YJOmnZBLWTnL9uHPmbAhxhg55kpiGdz1DQ6xOLsIg6Fv98tiYt0QDBrVmk0sNSIDMXIuRZfQ3pzkl+WPox1Ombrnh/PSz+Q=="}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:19.804484 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-9c812db1-1115-4190-b531-376c819c5115 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Getting quotas for project 1baa6ea2e19747d1af1e1042c362895d. Resources: {'key_pairs'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:42:19.809097 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-9c812db1-1115-4190-b531-376c819c5115 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Getting quotas for user 3ae56fd1172f4bf8806eefc6e8877fef and project 1baa6ea2e19747d1af1e1042c362895d. Resources: {'key_pairs'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:42:19.845093 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9c812db1-1115-4190-b531-376c819c5115 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.062796 Jul 11 14:42:19.845615 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 115/254] 10.222.0.72 () {60 vars in 1158 bytes} [Mon Jul 11 14:42:19 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:42:20.109290 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f646ca61-3739-4ae6-9742-8d243cb5084e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:20.109902 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f646ca61-3739-4ae6-9742-8d243cb5084e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:20.116286 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f646ca61-3739-4ae6-9742-8d243cb5084e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:20.116962 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f646ca61-3739-4ae6-9742-8d243cb5084e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:20.118598 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f646ca61-3739-4ae6-9742-8d243cb5084e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:20.120196 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f646ca61-3739-4ae6-9742-8d243cb5084e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:20.248504 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-14b2822e-ae25-412d-9630-1e01e1638403 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:20.249135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-14b2822e-ae25-412d-9630-1e01e1638403 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:20.257121 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-14b2822e-ae25-412d-9630-1e01e1638403 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:20.262231 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-14b2822e-ae25-412d-9630-1e01e1638403 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:20.263562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-14b2822e-ae25-412d-9630-1e01e1638403 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:20.264018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-14b2822e-ae25-412d-9630-1e01e1638403 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:20.268839 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:20.277080 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f646ca61-3739-4ae6-9742-8d243cb5084e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:20.286709 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f646ca61-3739-4ae6-9742-8d243cb5084e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:20.289631 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f646ca61-3739-4ae6-9742-8d243cb5084e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:20.291292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f646ca61-3739-4ae6-9742-8d243cb5084e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.186056 Jul 11 14:42:20.292413 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 140/255] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:20 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:42:20.515814 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-14b2822e-ae25-412d-9630-1e01e1638403 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:20.523463 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-14b2822e-ae25-412d-9630-1e01e1638403 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.278830 Jul 11 14:42:20.524052 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 116/256] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:20 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:21.302790 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-32692e1e-c618-4d95-8bbd-631c9e988c65 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1105407610", "imageRef": "a588fa24-23b5-4e53-87c6-7e600ee3a5ef", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8679b039-4e4e-4ae1-83e8-21c78c675a96"}], "security_groups": [{"name": "tempest-securitygroup--75469248"}], "key_name": "tempest-keypair-1046205418", "user_data": "}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:21.311608 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-0684e8f6-e8a8-4d98-b87e-c4d808137932 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:21.312365 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-0684e8f6-e8a8-4d98-b87e-c4d808137932 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:21.318619 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0684e8f6-e8a8-4d98-b87e-c4d808137932 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:21.319080 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0684e8f6-e8a8-4d98-b87e-c4d808137932 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:21.319882 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0684e8f6-e8a8-4d98-b87e-c4d808137932 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:21.320345 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0684e8f6-e8a8-4d98-b87e-c4d808137932 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:21.405265 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.network.neutron [None req-32692e1e-c618-4d95-8bbd-631c9e988c65 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] validate_networks() for [('8679b039-4e4e-4ae1-83e8-21c78c675a96', None, None, None, None, None)] {{(pid=130159) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:42:21.462171 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-0684e8f6-e8a8-4d98-b87e-c4d808137932 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:21.471010 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-0684e8f6-e8a8-4d98-b87e-c4d808137932 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:21.472631 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-0684e8f6-e8a8-4d98-b87e-c4d808137932 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:21.473970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-0684e8f6-e8a8-4d98-b87e-c4d808137932 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.166215 Jul 11 14:42:21.474604 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 117/257] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:21 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:42:21.541937 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-269f6576-2351-48ba-b934-47d51917334c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:21.543385 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-269f6576-2351-48ba-b934-47d51917334c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:21.549887 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-269f6576-2351-48ba-b934-47d51917334c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:21.550463 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-269f6576-2351-48ba-b934-47d51917334c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:21.551307 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-269f6576-2351-48ba-b934-47d51917334c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:21.551800 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-269f6576-2351-48ba-b934-47d51917334c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:21.723608 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-269f6576-2351-48ba-b934-47d51917334c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:21.734951 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-269f6576-2351-48ba-b934-47d51917334c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.198884 Jul 11 14:42:21.735546 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 118/258] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:21 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:21.815591 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-32692e1e-c618-4d95-8bbd-631c9e988c65 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Flavor limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:42:21.816041 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-32692e1e-c618-4d95-8bbd-631c9e988c65 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Image limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:42:21.816575 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-32692e1e-c618-4d95-8bbd-631c9e988c65 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Flavor pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:42:21.817101 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-32692e1e-c618-4d95-8bbd-631c9e988c65 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Image pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:42:21.817602 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-32692e1e-c618-4d95-8bbd-631c9e988c65 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:42:21.828235 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-32692e1e-c618-4d95-8bbd-631c9e988c65 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:21.828788 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-32692e1e-c618-4d95-8bbd-631c9e988c65 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:21.829642 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-32692e1e-c618-4d95-8bbd-631c9e988c65 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:21.830277 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-32692e1e-c618-4d95-8bbd-631c9e988c65 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:21.992370 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-32692e1e-c618-4d95-8bbd-631c9e988c65 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Getting quotas for project 1baa6ea2e19747d1af1e1042c362895d. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:42:21.996501 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-32692e1e-c618-4d95-8bbd-631c9e988c65 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Getting quotas for user 3ae56fd1172f4bf8806eefc6e8877fef and project 1baa6ea2e19747d1af1e1042c362895d. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:42:22.007384 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-32692e1e-c618-4d95-8bbd-631c9e988c65 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Going to run 1 instances... {{(pid=130159) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:42:22.009927 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-32692e1e-c618-4d95-8bbd-631c9e988c65 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a588fa24-23b5-4e53-87c6-7e600ee3a5ef',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130159) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:42:22.055830 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-32692e1e-c618-4d95-8bbd-631c9e988c65 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.1 time: 0.757287 Jul 11 14:42:22.056518 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 141/259] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:42:21 2022] POST /compute/v2.1/servers => generated 400 bytes in 759 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 11 14:42:22.066572 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d227aff0-3259-48c8-8d7f-f2e4400ce634 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:22.067216 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d227aff0-3259-48c8-8d7f-f2e4400ce634 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:22.125403 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d227aff0-3259-48c8-8d7f-f2e4400ce634 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:22.136305 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d227aff0-3259-48c8-8d7f-f2e4400ce634 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1242 microversion: 2.1 time: 0.073478 Jul 11 14:42:22.136926 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 119/260] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:22 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1242 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:22.493434 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e41a09fb-3216-4ca0-a79c-9b2802dd6091 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:22.494668 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e41a09fb-3216-4ca0-a79c-9b2802dd6091 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:22.501521 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e41a09fb-3216-4ca0-a79c-9b2802dd6091 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:22.502173 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e41a09fb-3216-4ca0-a79c-9b2802dd6091 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:22.503237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e41a09fb-3216-4ca0-a79c-9b2802dd6091 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:22.504015 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e41a09fb-3216-4ca0-a79c-9b2802dd6091 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:22.664618 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e41a09fb-3216-4ca0-a79c-9b2802dd6091 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:22.673786 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e41a09fb-3216-4ca0-a79c-9b2802dd6091 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:22.675878 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e41a09fb-3216-4ca0-a79c-9b2802dd6091 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:22.677814 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e41a09fb-3216-4ca0-a79c-9b2802dd6091 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.187751 Jul 11 14:42:22.678072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 142/261] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:22 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:42:22.754972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-02bde304-e507-4ee7-afe9-48cc67a63fde tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:22.755573 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-02bde304-e507-4ee7-afe9-48cc67a63fde tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:22.764202 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-02bde304-e507-4ee7-afe9-48cc67a63fde tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:22.764663 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-02bde304-e507-4ee7-afe9-48cc67a63fde tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:22.765588 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-02bde304-e507-4ee7-afe9-48cc67a63fde tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:22.766029 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-02bde304-e507-4ee7-afe9-48cc67a63fde tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:22.904859 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-02bde304-e507-4ee7-afe9-48cc67a63fde tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:22.912369 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-02bde304-e507-4ee7-afe9-48cc67a63fde tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.162937 Jul 11 14:42:22.912955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 120/262] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:22 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:23.153418 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9d974429-8d2c-4d8a-9317-3ab7acfdfaf2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:23.154037 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9d974429-8d2c-4d8a-9317-3ab7acfdfaf2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:23.160352 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9d974429-8d2c-4d8a-9317-3ab7acfdfaf2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:23.160806 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9d974429-8d2c-4d8a-9317-3ab7acfdfaf2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:23.161566 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9d974429-8d2c-4d8a-9317-3ab7acfdfaf2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:23.162000 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9d974429-8d2c-4d8a-9317-3ab7acfdfaf2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:23.182301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:23.236324 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9d974429-8d2c-4d8a-9317-3ab7acfdfaf2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:23.244420 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9d974429-8d2c-4d8a-9317-3ab7acfdfaf2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1302 microversion: 2.1 time: 0.094893 Jul 11 14:42:23.244996 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 143/263] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:23 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1302 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:23.696781 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2847c383-efcf-4390-aaa5-20d0797bfaa8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:23.697472 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2847c383-efcf-4390-aaa5-20d0797bfaa8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:23.704474 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2847c383-efcf-4390-aaa5-20d0797bfaa8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:23.704932 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2847c383-efcf-4390-aaa5-20d0797bfaa8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:23.705960 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2847c383-efcf-4390-aaa5-20d0797bfaa8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:23.706504 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2847c383-efcf-4390-aaa5-20d0797bfaa8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:23.922073 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2847c383-efcf-4390-aaa5-20d0797bfaa8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:23.929613 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1995bc3e-f8ef-441c-8027-84ff4e5fa759 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:23.930308 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1995bc3e-f8ef-441c-8027-84ff4e5fa759 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:23.943401 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1995bc3e-f8ef-441c-8027-84ff4e5fa759 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:23.944273 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1995bc3e-f8ef-441c-8027-84ff4e5fa759 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:23.948374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2847c383-efcf-4390-aaa5-20d0797bfaa8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:23.949702 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1995bc3e-f8ef-441c-8027-84ff4e5fa759 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:23.950155 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1995bc3e-f8ef-441c-8027-84ff4e5fa759 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:23.953982 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2847c383-efcf-4390-aaa5-20d0797bfaa8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:23.955225 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2847c383-efcf-4390-aaa5-20d0797bfaa8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.262483 Jul 11 14:42:23.956054 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 121/264] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:23 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:42:24.112049 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1995bc3e-f8ef-441c-8027-84ff4e5fa759 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:24.120480 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1995bc3e-f8ef-441c-8027-84ff4e5fa759 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.194900 Jul 11 14:42:24.121256 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 144/265] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:23 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:24.262189 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-4f78f522-9544-4d0a-9a07-52dff422eb85 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:24.262771 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-4f78f522-9544-4d0a-9a07-52dff422eb85 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:24.270245 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4f78f522-9544-4d0a-9a07-52dff422eb85 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:24.271216 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4f78f522-9544-4d0a-9a07-52dff422eb85 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:24.275296 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4f78f522-9544-4d0a-9a07-52dff422eb85 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:24.275780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4f78f522-9544-4d0a-9a07-52dff422eb85 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:24.481902 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4f78f522-9544-4d0a-9a07-52dff422eb85 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:24.489238 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-4f78f522-9544-4d0a-9a07-52dff422eb85 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1366 microversion: 2.1 time: 0.231579 Jul 11 14:42:24.489813 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 122/266] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:24 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1366 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:24.870200 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-0642e63b-d3da-4633-bf1d-19508ad91d50 req-8f2ff993-790c-49a9-9dde-edf4608c4220 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7a49d1ca-7491-41e0-b0d8-37c92b9d3562", "tag": "c62c0fe9-fd78-4a72-aef8-9bd42e6d81de"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:24.880830 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-0642e63b-d3da-4633-bf1d-19508ad91d50 req-8f2ff993-790c-49a9-9dde-edf4608c4220 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:24.881520 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-0642e63b-d3da-4633-bf1d-19508ad91d50 req-8f2ff993-790c-49a9-9dde-edf4608c4220 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:24.901042 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-0642e63b-d3da-4633-bf1d-19508ad91d50 req-8f2ff993-790c-49a9-9dde-edf4608c4220 service nova] Creating event network-changed:c62c0fe9-fd78-4a72-aef8-9bd42e6d81de for instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 on hv-cinder-maste Jul 11 14:42:24.912033 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-0642e63b-d3da-4633-bf1d-19508ad91d50 req-8f2ff993-790c-49a9-9dde-edf4608c4220 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046059 Jul 11 14:42:24.913255 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 145/267] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:42:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:42:24.973815 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-cbd3d492-33e5-49f8-b427-826153979e2f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:24.974474 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-cbd3d492-33e5-49f8-b427-826153979e2f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:24.982646 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cbd3d492-33e5-49f8-b427-826153979e2f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:24.983129 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cbd3d492-33e5-49f8-b427-826153979e2f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:24.983909 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cbd3d492-33e5-49f8-b427-826153979e2f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:24.984510 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cbd3d492-33e5-49f8-b427-826153979e2f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:25.138845 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1ff649b5-30bb-45fe-aa73-b89fea3792c3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:25.140447 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1ff649b5-30bb-45fe-aa73-b89fea3792c3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:25.159718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1ff649b5-30bb-45fe-aa73-b89fea3792c3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:25.159903 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1ff649b5-30bb-45fe-aa73-b89fea3792c3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:25.160888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1ff649b5-30bb-45fe-aa73-b89fea3792c3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:25.161495 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1ff649b5-30bb-45fe-aa73-b89fea3792c3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:25.320730 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1ff649b5-30bb-45fe-aa73-b89fea3792c3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:25.328611 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1ff649b5-30bb-45fe-aa73-b89fea3792c3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.194403 Jul 11 14:42:25.329283 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 146/268] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:25 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:25.353137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-6330ec19-9577-4d63-ab64-1420a6b0b694 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1229234174", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-2098849656", "security_groups": [{"name": "tempest-secgroup-smoke-2075789476"}], "block_device_mapping_v2": [{"uuid": "ccbe23b1-2e12-4e18-9982-7c804ff58b5f", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "e7191ae9-3a75-447f-ae05-2827e78eae91"}]}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:25.376356 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-cbd3d492-33e5-49f8-b427-826153979e2f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:25.384963 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-cbd3d492-33e5-49f8-b427-826153979e2f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:25.386824 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-cbd3d492-33e5-49f8-b427-826153979e2f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:25.388307 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-cbd3d492-33e5-49f8-b427-826153979e2f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.418628 Jul 11 14:42:25.396951 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 123/269] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:24 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:42:25.491619 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.network.neutron [None req-6330ec19-9577-4d63-ab64-1420a6b0b694 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] validate_networks() for [('e7191ae9-3a75-447f-ae05-2827e78eae91', None, None, None, None, None)] {{(pid=130159) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:42:25.505803 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-27789af3-12b3-438a-8024-364a1886e297 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:25.506416 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-27789af3-12b3-438a-8024-364a1886e297 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:25.512732 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-27789af3-12b3-438a-8024-364a1886e297 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:25.513182 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-27789af3-12b3-438a-8024-364a1886e297 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:25.513950 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-27789af3-12b3-438a-8024-364a1886e297 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:25.514395 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-27789af3-12b3-438a-8024-364a1886e297 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:25.665384 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-27789af3-12b3-438a-8024-364a1886e297 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:25.679371 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-27789af3-12b3-438a-8024-364a1886e297 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1366 microversion: 2.1 time: 0.176344 Jul 11 14:42:25.679371 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 124/270] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:25 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1366 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:25.898095 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-6330ec19-9577-4d63-ab64-1420a6b0b694 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Flavor limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:42:25.899124 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-6330ec19-9577-4d63-ab64-1420a6b0b694 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Image limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:42:25.899786 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-6330ec19-9577-4d63-ab64-1420a6b0b694 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Flavor pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:42:25.900380 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-6330ec19-9577-4d63-ab64-1420a6b0b694 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Image pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:42:25.900788 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-6330ec19-9577-4d63-ab64-1420a6b0b694 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:42:25.909582 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6330ec19-9577-4d63-ab64-1420a6b0b694 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:25.910044 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6330ec19-9577-4d63-ab64-1420a6b0b694 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:25.910869 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6330ec19-9577-4d63-ab64-1420a6b0b694 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:25.911327 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6330ec19-9577-4d63-ab64-1420a6b0b694 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:25.928123 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-6330ec19-9577-4d63-ab64-1420a6b0b694 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Getting quotas for project 3d541b7a72dc4b4494a42bda6fa75026. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:42:25.932282 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-6330ec19-9577-4d63-ab64-1420a6b0b694 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Getting quotas for user 0f8e300cd96e446db7430044e69d997a and project 3d541b7a72dc4b4494a42bda6fa75026. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:42:25.989724 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-6330ec19-9577-4d63-ab64-1420a6b0b694 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Going to run 1 instances... {{(pid=130159) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:42:25.991979 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-6330ec19-9577-4d63-ab64-1420a6b0b694 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='ccbe23b1-2e12-4e18-9982-7c804ff58b5f',volume_size=None,volume_type=None)] {{(pid=130159) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:42:26.130292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-6330ec19-9577-4d63-ab64-1420a6b0b694 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 0.781768 Jul 11 14:42:26.131026 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 147/271] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:42:25 2022] POST /compute/v2.1/servers => generated 406 bytes in 783 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 11 14:42:26.142281 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e9b4cd81-2a1a-4d8e-95e7-722a99ba5fc0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:26.143170 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-e9b4cd81-2a1a-4d8e-95e7-722a99ba5fc0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:26.280950 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e9b4cd81-2a1a-4d8e-95e7-722a99ba5fc0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:26.289340 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e9b4cd81-2a1a-4d8e-95e7-722a99ba5fc0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1095 microversion: 2.1 time: 0.149895 Jul 11 14:42:26.290409 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 125/272] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:26 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1095 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:26.345659 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1e25b7b0-e317-474d-bc85-f01b995e3b48 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:26.346481 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1e25b7b0-e317-474d-bc85-f01b995e3b48 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:26.353418 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1e25b7b0-e317-474d-bc85-f01b995e3b48 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:26.354011 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1e25b7b0-e317-474d-bc85-f01b995e3b48 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:26.355132 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1e25b7b0-e317-474d-bc85-f01b995e3b48 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:26.355845 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1e25b7b0-e317-474d-bc85-f01b995e3b48 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:26.406814 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a98cedb2-3067-49ea-afa1-8c9575c29329 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:26.407435 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a98cedb2-3067-49ea-afa1-8c9575c29329 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:26.420939 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a98cedb2-3067-49ea-afa1-8c9575c29329 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:26.425373 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a98cedb2-3067-49ea-afa1-8c9575c29329 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:26.426711 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a98cedb2-3067-49ea-afa1-8c9575c29329 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:26.427985 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a98cedb2-3067-49ea-afa1-8c9575c29329 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:26.547533 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1e25b7b0-e317-474d-bc85-f01b995e3b48 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:26.560364 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1e25b7b0-e317-474d-bc85-f01b995e3b48 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.218669 Jul 11 14:42:26.561236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 148/273] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:26 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:26.699852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-0a485c18-08f2-4bd8-b5fc-d9b8fd360d97 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:26.700994 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-0a485c18-08f2-4bd8-b5fc-d9b8fd360d97 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:26.709131 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0a485c18-08f2-4bd8-b5fc-d9b8fd360d97 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:26.709672 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0a485c18-08f2-4bd8-b5fc-d9b8fd360d97 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:26.710558 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0a485c18-08f2-4bd8-b5fc-d9b8fd360d97 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:26.711754 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0a485c18-08f2-4bd8-b5fc-d9b8fd360d97 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:26.735091 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a98cedb2-3067-49ea-afa1-8c9575c29329 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:26.748158 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a98cedb2-3067-49ea-afa1-8c9575c29329 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:26.749689 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a98cedb2-3067-49ea-afa1-8c9575c29329 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:26.753137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a98cedb2-3067-49ea-afa1-8c9575c29329 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.348374 Jul 11 14:42:26.753738 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 126/274] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:26 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:42:26.870027 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0a485c18-08f2-4bd8-b5fc-d9b8fd360d97 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:26.878468 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-0a485c18-08f2-4bd8-b5fc-d9b8fd360d97 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1366 microversion: 2.1 time: 0.184616 Jul 11 14:42:26.879254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 149/275] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:26 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1366 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:27.311448 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-01771197-e699-49c8-b230-c91667df37a8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:27.312845 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-01771197-e699-49c8-b230-c91667df37a8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:27.330237 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-01771197-e699-49c8-b230-c91667df37a8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:27.331540 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-01771197-e699-49c8-b230-c91667df37a8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:27.332357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-01771197-e699-49c8-b230-c91667df37a8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:27.332801 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-01771197-e699-49c8-b230-c91667df37a8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:27.389069 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-54e78645-4b23-4d5d-bd71-5535a36dce7b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:27.396277 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-54e78645-4b23-4d5d-bd71-5535a36dce7b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:27.396915 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-54e78645-4b23-4d5d-bd71-5535a36dce7b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:27.411611 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-01771197-e699-49c8-b230-c91667df37a8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:27.420915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-01771197-e699-49c8-b230-c91667df37a8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1201 microversion: 2.1 time: 0.118125 Jul 11 14:42:27.421741 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 127/276] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:27 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1201 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:27.471056 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-54e78645-4b23-4d5d-bd71-5535a36dce7b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Going to try to stop instance {{(pid=130159) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 11 14:42:27.522118 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-54e78645-4b23-4d5d-bd71-5535a36dce7b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "POST /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d/action" status: 202 len: 0 microversion: 2.1 time: 0.136749 Jul 11 14:42:27.526934 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 150/277] 10.222.0.72 () {60 vars in 1277 bytes} [Mon Jul 11 14:42:27 2022] POST /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d/action => generated 0 bytes in 143 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 11 14:42:27.534347 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-58488741-f978-4700-a09a-d7bbffbc6239 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:27.538698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-58488741-f978-4700-a09a-d7bbffbc6239 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:27.547462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-58488741-f978-4700-a09a-d7bbffbc6239 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:27.548124 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-58488741-f978-4700-a09a-d7bbffbc6239 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:27.549109 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-58488741-f978-4700-a09a-d7bbffbc6239 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:27.549761 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-58488741-f978-4700-a09a-d7bbffbc6239 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:27.579672 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8a663dbb-ff94-48df-91bb-8ec7283ba80d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:27.580385 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8a663dbb-ff94-48df-91bb-8ec7283ba80d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:27.587679 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8a663dbb-ff94-48df-91bb-8ec7283ba80d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:27.588133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8a663dbb-ff94-48df-91bb-8ec7283ba80d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:27.588915 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8a663dbb-ff94-48df-91bb-8ec7283ba80d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:27.589403 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8a663dbb-ff94-48df-91bb-8ec7283ba80d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:27.704106 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-58488741-f978-4700-a09a-d7bbffbc6239 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:27.712296 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-58488741-f978-4700-a09a-d7bbffbc6239 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1722 microversion: 2.1 time: 0.185318 Jul 11 14:42:27.712929 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 128/278] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:27 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1722 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:27.771548 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-521ec4a6-4d28-4bed-be5c-ff755cc230d7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:27.772313 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-521ec4a6-4d28-4bed-be5c-ff755cc230d7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:27.779227 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-521ec4a6-4d28-4bed-be5c-ff755cc230d7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:27.780109 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-521ec4a6-4d28-4bed-be5c-ff755cc230d7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:27.780879 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-521ec4a6-4d28-4bed-be5c-ff755cc230d7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:27.781316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-521ec4a6-4d28-4bed-be5c-ff755cc230d7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:27.830099 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8a663dbb-ff94-48df-91bb-8ec7283ba80d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:27.839603 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8a663dbb-ff94-48df-91bb-8ec7283ba80d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.265753 Jul 11 14:42:27.846931 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 151/279] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:27 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:27.897081 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3a511ab9-dd65-4b49-b68b-6a9f20ce720e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:27.897775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3a511ab9-dd65-4b49-b68b-6a9f20ce720e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:27.904684 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3a511ab9-dd65-4b49-b68b-6a9f20ce720e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:27.905142 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3a511ab9-dd65-4b49-b68b-6a9f20ce720e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:27.905945 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3a511ab9-dd65-4b49-b68b-6a9f20ce720e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:27.906385 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3a511ab9-dd65-4b49-b68b-6a9f20ce720e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:27.931953 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-521ec4a6-4d28-4bed-be5c-ff755cc230d7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:27.945307 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-521ec4a6-4d28-4bed-be5c-ff755cc230d7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:27.947170 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-521ec4a6-4d28-4bed-be5c-ff755cc230d7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:27.948339 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-521ec4a6-4d28-4bed-be5c-ff755cc230d7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.180860 Jul 11 14:42:27.948890 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 129/280] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:27 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:42:28.109720 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3a511ab9-dd65-4b49-b68b-6a9f20ce720e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:28.121446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3a511ab9-dd65-4b49-b68b-6a9f20ce720e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1366 microversion: 2.1 time: 0.228138 Jul 11 14:42:28.133294 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 152/281] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:27 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1366 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:28.438569 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c25084a9-bbae-41eb-aed2-b7a1771f9088 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:28.439819 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c25084a9-bbae-41eb-aed2-b7a1771f9088 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:28.447041 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c25084a9-bbae-41eb-aed2-b7a1771f9088 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:28.447493 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c25084a9-bbae-41eb-aed2-b7a1771f9088 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:28.448265 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c25084a9-bbae-41eb-aed2-b7a1771f9088 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:28.448703 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c25084a9-bbae-41eb-aed2-b7a1771f9088 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:28.600137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c25084a9-bbae-41eb-aed2-b7a1771f9088 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:28.608763 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c25084a9-bbae-41eb-aed2-b7a1771f9088 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1267 microversion: 2.1 time: 0.174152 Jul 11 14:42:28.609380 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 130/282] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:28 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1267 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:28.731114 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a00c57ee-00f6-4831-966d-6eb165123a4a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:28.732077 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a00c57ee-00f6-4831-966d-6eb165123a4a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:28.739209 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a00c57ee-00f6-4831-966d-6eb165123a4a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:28.739667 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a00c57ee-00f6-4831-966d-6eb165123a4a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:28.740443 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a00c57ee-00f6-4831-966d-6eb165123a4a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:28.740887 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a00c57ee-00f6-4831-966d-6eb165123a4a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:28.887065 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-70319222-de14-482e-a1d4-586e705a0471 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:28.887792 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-70319222-de14-482e-a1d4-586e705a0471 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:28.894252 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-70319222-de14-482e-a1d4-586e705a0471 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:28.894738 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-70319222-de14-482e-a1d4-586e705a0471 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:28.895516 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-70319222-de14-482e-a1d4-586e705a0471 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:28.895986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-70319222-de14-482e-a1d4-586e705a0471 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:28.962608 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a00c57ee-00f6-4831-966d-6eb165123a4a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:28.971870 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a00c57ee-00f6-4831-966d-6eb165123a4a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1722 microversion: 2.1 time: 0.245090 Jul 11 14:42:28.983243 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 153/283] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:28 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1722 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:28.988396 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-190342c9-0895-4f6f-a58a-25de688125b9 req-f797aec1-6dae-4577-b38c-67720a599dbb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3a91aed4-52ff-4220-bc8f-571c50452865", "tag": "957d5045-a8c1-4d28-8ee4-755331ae2c84"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:28.998237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-190342c9-0895-4f6f-a58a-25de688125b9 req-f797aec1-6dae-4577-b38c-67720a599dbb service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:28.998956 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-190342c9-0895-4f6f-a58a-25de688125b9 req-f797aec1-6dae-4577-b38c-67720a599dbb service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:29.013803 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-190342c9-0895-4f6f-a58a-25de688125b9 req-f797aec1-6dae-4577-b38c-67720a599dbb service nova] Creating event network-changed:957d5045-a8c1-4d28-8ee4-755331ae2c84 for instance 3a91aed4-52ff-4220-bc8f-571c50452865 on hv-cinder-maste Jul 11 14:42:29.019764 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-190342c9-0895-4f6f-a58a-25de688125b9 req-f797aec1-6dae-4577-b38c-67720a599dbb service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034852 Jul 11 14:42:29.020692 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 154/284] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:42:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:42:29.027332 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9f27702e-54d5-414c-a2f5-a5e6ccc63ce3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:29.029337 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9f27702e-54d5-414c-a2f5-a5e6ccc63ce3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:29.037065 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9f27702e-54d5-414c-a2f5-a5e6ccc63ce3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:29.037702 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9f27702e-54d5-414c-a2f5-a5e6ccc63ce3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:29.038771 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9f27702e-54d5-414c-a2f5-a5e6ccc63ce3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:29.039871 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9f27702e-54d5-414c-a2f5-a5e6ccc63ce3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:29.076941 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-70319222-de14-482e-a1d4-586e705a0471 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:29.087676 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-70319222-de14-482e-a1d4-586e705a0471 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.230509 Jul 11 14:42:29.088283 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 131/285] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:28 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:29.141486 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2f7e54a5-2b3b-4c6a-993b-34ce51dd00a8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:29.142109 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2f7e54a5-2b3b-4c6a-993b-34ce51dd00a8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:29.155094 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2f7e54a5-2b3b-4c6a-993b-34ce51dd00a8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:29.155554 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2f7e54a5-2b3b-4c6a-993b-34ce51dd00a8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:29.156335 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2f7e54a5-2b3b-4c6a-993b-34ce51dd00a8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:29.156855 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2f7e54a5-2b3b-4c6a-993b-34ce51dd00a8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:29.251195 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9f27702e-54d5-414c-a2f5-a5e6ccc63ce3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:29.260931 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9f27702e-54d5-414c-a2f5-a5e6ccc63ce3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:29.263212 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9f27702e-54d5-414c-a2f5-a5e6ccc63ce3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:29.264816 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9f27702e-54d5-414c-a2f5-a5e6ccc63ce3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.242489 Jul 11 14:42:29.266233 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 155/286] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:29 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:42:29.301058 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2f7e54a5-2b3b-4c6a-993b-34ce51dd00a8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:29.309050 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2f7e54a5-2b3b-4c6a-993b-34ce51dd00a8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1366 microversion: 2.1 time: 0.171584 Jul 11 14:42:29.309599 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 132/287] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:29 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1366 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:29.626365 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f4bee83e-fc52-425a-9370-0ab5065953bc tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:29.627007 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f4bee83e-fc52-425a-9370-0ab5065953bc tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:29.633179 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f4bee83e-fc52-425a-9370-0ab5065953bc tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:29.633673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f4bee83e-fc52-425a-9370-0ab5065953bc tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:29.634472 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f4bee83e-fc52-425a-9370-0ab5065953bc tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:29.634940 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f4bee83e-fc52-425a-9370-0ab5065953bc tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:29.781574 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f4bee83e-fc52-425a-9370-0ab5065953bc tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:29.790910 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f4bee83e-fc52-425a-9370-0ab5065953bc tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1267 microversion: 2.1 time: 0.168206 Jul 11 14:42:29.791538 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 156/288] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:29 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1267 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:29.990300 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-98a67ea8-5fdd-4b37-a867-3622fd21e4cd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:29.991134 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-98a67ea8-5fdd-4b37-a867-3622fd21e4cd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:29.998533 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-98a67ea8-5fdd-4b37-a867-3622fd21e4cd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:29.999017 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-98a67ea8-5fdd-4b37-a867-3622fd21e4cd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:29.999946 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-98a67ea8-5fdd-4b37-a867-3622fd21e4cd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:30.000407 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-98a67ea8-5fdd-4b37-a867-3622fd21e4cd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:30.107173 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c886f778-c0e7-40cc-84aa-bb7b3a7dbe0f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:30.107798 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c886f778-c0e7-40cc-84aa-bb7b3a7dbe0f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:30.115636 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c886f778-c0e7-40cc-84aa-bb7b3a7dbe0f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:30.116088 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c886f778-c0e7-40cc-84aa-bb7b3a7dbe0f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:30.116869 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c886f778-c0e7-40cc-84aa-bb7b3a7dbe0f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:30.117838 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c886f778-c0e7-40cc-84aa-bb7b3a7dbe0f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:30.158773 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-98a67ea8-5fdd-4b37-a867-3622fd21e4cd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:30.166678 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-98a67ea8-5fdd-4b37-a867-3622fd21e4cd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1722 microversion: 2.1 time: 0.180265 Jul 11 14:42:30.167248 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 133/289] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:29 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1722 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:30.284289 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8384e190-beb9-4d93-8be6-598ecad5b41d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:30.284910 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-8384e190-beb9-4d93-8be6-598ecad5b41d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:30.292322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8384e190-beb9-4d93-8be6-598ecad5b41d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:30.292771 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8384e190-beb9-4d93-8be6-598ecad5b41d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:30.293533 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8384e190-beb9-4d93-8be6-598ecad5b41d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:30.293961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8384e190-beb9-4d93-8be6-598ecad5b41d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:30.336382 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c886f778-c0e7-40cc-84aa-bb7b3a7dbe0f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:30.343974 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c886f778-c0e7-40cc-84aa-bb7b3a7dbe0f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.242900 Jul 11 14:42:30.344763 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 157/290] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:30 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:30.349390 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3b3c95b1-540e-40e3-aa6a-515de945194e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:30.350209 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3b3c95b1-540e-40e3-aa6a-515de945194e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:30.357177 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3b3c95b1-540e-40e3-aa6a-515de945194e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:30.357622 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3b3c95b1-540e-40e3-aa6a-515de945194e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:30.358379 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3b3c95b1-540e-40e3-aa6a-515de945194e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:30.358840 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3b3c95b1-540e-40e3-aa6a-515de945194e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:30.448212 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8384e190-beb9-4d93-8be6-598ecad5b41d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:30.449962 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=100 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:30.460301 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8384e190-beb9-4d93-8be6-598ecad5b41d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:30.460301 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8384e190-beb9-4d93-8be6-598ecad5b41d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:30.463664 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8384e190-beb9-4d93-8be6-598ecad5b41d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.183772 Jul 11 14:42:30.464235 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 134/291] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:30 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:42:30.527376 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3b3c95b1-540e-40e3-aa6a-515de945194e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:30.535897 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3b3c95b1-540e-40e3-aa6a-515de945194e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1366 microversion: 2.1 time: 0.190205 Jul 11 14:42:30.536485 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 158/292] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:30 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1366 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:30.808547 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9c68904b-ba38-4dd4-8632-bc2e65c34026 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:30.809290 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-9c68904b-ba38-4dd4-8632-bc2e65c34026 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:30.815639 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9c68904b-ba38-4dd4-8632-bc2e65c34026 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:30.816092 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9c68904b-ba38-4dd4-8632-bc2e65c34026 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:30.816883 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9c68904b-ba38-4dd4-8632-bc2e65c34026 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:30.817317 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9c68904b-ba38-4dd4-8632-bc2e65c34026 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:30.952914 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-9c68904b-ba38-4dd4-8632-bc2e65c34026 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:30.960772 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9c68904b-ba38-4dd4-8632-bc2e65c34026 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1267 microversion: 2.1 time: 0.156668 Jul 11 14:42:30.961344 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 135/293] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:30 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1267 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:31.185567 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3b0ade1d-80b8-4c3b-a401-6a71882c4b20 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:31.186134 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3b0ade1d-80b8-4c3b-a401-6a71882c4b20 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:31.192630 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3b0ade1d-80b8-4c3b-a401-6a71882c4b20 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:31.193083 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3b0ade1d-80b8-4c3b-a401-6a71882c4b20 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:31.193856 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3b0ade1d-80b8-4c3b-a401-6a71882c4b20 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:31.194295 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3b0ade1d-80b8-4c3b-a401-6a71882c4b20 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:31.323776 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3b0ade1d-80b8-4c3b-a401-6a71882c4b20 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:31.332928 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3b0ade1d-80b8-4c3b-a401-6a71882c4b20 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1722 microversion: 2.1 time: 0.151556 Jul 11 14:42:31.333778 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 159/294] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:31 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1722 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:31.360933 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-67a80fb3-4ccf-48af-b292-2f9f4cfd76de tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:31.361544 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-67a80fb3-4ccf-48af-b292-2f9f4cfd76de tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:31.369263 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-67a80fb3-4ccf-48af-b292-2f9f4cfd76de tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:31.369719 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-67a80fb3-4ccf-48af-b292-2f9f4cfd76de tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:31.370499 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-67a80fb3-4ccf-48af-b292-2f9f4cfd76de tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:31.370936 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-67a80fb3-4ccf-48af-b292-2f9f4cfd76de tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:31.482890 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-429d74ef-6c9b-406d-9311-22e0f56ece55 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:31.483654 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-429d74ef-6c9b-406d-9311-22e0f56ece55 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:31.490413 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-429d74ef-6c9b-406d-9311-22e0f56ece55 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:31.490990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-429d74ef-6c9b-406d-9311-22e0f56ece55 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:31.491821 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-429d74ef-6c9b-406d-9311-22e0f56ece55 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:31.492925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-429d74ef-6c9b-406d-9311-22e0f56ece55 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:31.512702 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-67a80fb3-4ccf-48af-b292-2f9f4cfd76de tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:31.520469 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-67a80fb3-4ccf-48af-b292-2f9f4cfd76de tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.163282 Jul 11 14:42:31.521046 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 136/295] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:31 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:31.554504 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-507b0afd-93d0-4278-be62-fb0fe508e266 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:31.555756 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-507b0afd-93d0-4278-be62-fb0fe508e266 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:31.562061 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-507b0afd-93d0-4278-be62-fb0fe508e266 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:31.562667 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-507b0afd-93d0-4278-be62-fb0fe508e266 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:31.563611 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-507b0afd-93d0-4278-be62-fb0fe508e266 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:31.564046 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-507b0afd-93d0-4278-be62-fb0fe508e266 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:31.628603 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-429d74ef-6c9b-406d-9311-22e0f56ece55 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:31.636609 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-429d74ef-6c9b-406d-9311-22e0f56ece55 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:31.638138 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-429d74ef-6c9b-406d-9311-22e0f56ece55 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:31.639357 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-429d74ef-6c9b-406d-9311-22e0f56ece55 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.161447 Jul 11 14:42:31.639927 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 160/296] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:31 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:42:31.694064 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-507b0afd-93d0-4278-be62-fb0fe508e266 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:31.701742 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-507b0afd-93d0-4278-be62-fb0fe508e266 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1366 microversion: 2.1 time: 0.151178 Jul 11 14:42:31.702312 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 137/297] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:31 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1366 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:31.977047 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-0efed1bd-2b24-4490-8dba-e009503f36db tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:31.977649 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-0efed1bd-2b24-4490-8dba-e009503f36db tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:31.984064 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0efed1bd-2b24-4490-8dba-e009503f36db tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:31.984508 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0efed1bd-2b24-4490-8dba-e009503f36db tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:31.985249 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0efed1bd-2b24-4490-8dba-e009503f36db tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:31.985679 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0efed1bd-2b24-4490-8dba-e009503f36db tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:31.989657 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:32.122622 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0efed1bd-2b24-4490-8dba-e009503f36db tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:32.130054 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-0efed1bd-2b24-4490-8dba-e009503f36db tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1267 microversion: 2.1 time: 0.156824 Jul 11 14:42:32.130644 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 161/298] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:31 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1267 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:32.349347 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-939df48f-9983-42b9-ac34-c7db85a6bab5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:32.349956 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-939df48f-9983-42b9-ac34-c7db85a6bab5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:32.357346 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-939df48f-9983-42b9-ac34-c7db85a6bab5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:32.357765 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-939df48f-9983-42b9-ac34-c7db85a6bab5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:32.359076 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-939df48f-9983-42b9-ac34-c7db85a6bab5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:32.359076 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-939df48f-9983-42b9-ac34-c7db85a6bab5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:32.496758 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-939df48f-9983-42b9-ac34-c7db85a6bab5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:32.504757 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-939df48f-9983-42b9-ac34-c7db85a6bab5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1722 microversion: 2.1 time: 0.159165 Jul 11 14:42:32.505330 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 138/299] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:32 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1722 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:32.537403 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-758325ec-b0ae-4208-91ef-d421854b2564 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:32.538040 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-758325ec-b0ae-4208-91ef-d421854b2564 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:32.544120 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-758325ec-b0ae-4208-91ef-d421854b2564 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:32.544569 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-758325ec-b0ae-4208-91ef-d421854b2564 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:32.545321 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-758325ec-b0ae-4208-91ef-d421854b2564 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:32.545757 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-758325ec-b0ae-4208-91ef-d421854b2564 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:32.657726 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-4e3c4822-ff6e-412e-9fc8-3b0e0eae4dfe tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:32.658387 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-4e3c4822-ff6e-412e-9fc8-3b0e0eae4dfe tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:32.664466 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4e3c4822-ff6e-412e-9fc8-3b0e0eae4dfe tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:32.665019 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4e3c4822-ff6e-412e-9fc8-3b0e0eae4dfe tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:32.665777 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4e3c4822-ff6e-412e-9fc8-3b0e0eae4dfe tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:32.666316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4e3c4822-ff6e-412e-9fc8-3b0e0eae4dfe tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:32.678672 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-758325ec-b0ae-4208-91ef-d421854b2564 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:32.686582 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-758325ec-b0ae-4208-91ef-d421854b2564 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.153002 Jul 11 14:42:32.687149 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 162/300] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:32 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:32.720097 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a4c9a76e-88b3-450c-b079-0295262a6527 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:32.720731 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a4c9a76e-88b3-450c-b079-0295262a6527 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:32.727376 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a4c9a76e-88b3-450c-b079-0295262a6527 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:32.727820 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a4c9a76e-88b3-450c-b079-0295262a6527 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:32.728595 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a4c9a76e-88b3-450c-b079-0295262a6527 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:32.729026 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a4c9a76e-88b3-450c-b079-0295262a6527 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:32.801749 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4e3c4822-ff6e-412e-9fc8-3b0e0eae4dfe tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:32.810594 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4e3c4822-ff6e-412e-9fc8-3b0e0eae4dfe tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:32.812109 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4e3c4822-ff6e-412e-9fc8-3b0e0eae4dfe tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:32.813269 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-4e3c4822-ff6e-412e-9fc8-3b0e0eae4dfe tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.159565 Jul 11 14:42:32.813826 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 139/301] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:32 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:42:32.870132 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a4c9a76e-88b3-450c-b079-0295262a6527 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:32.879529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a4c9a76e-88b3-450c-b079-0295262a6527 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1366 microversion: 2.1 time: 0.163296 Jul 11 14:42:32.879985 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 163/302] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:32 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1366 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:33.146834 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a66278f1-0885-4b33-bc7b-823e08e0a089 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:33.147437 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a66278f1-0885-4b33-bc7b-823e08e0a089 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:33.154020 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a66278f1-0885-4b33-bc7b-823e08e0a089 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:33.154495 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a66278f1-0885-4b33-bc7b-823e08e0a089 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:33.156732 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a66278f1-0885-4b33-bc7b-823e08e0a089 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:33.157176 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a66278f1-0885-4b33-bc7b-823e08e0a089 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:33.294262 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a66278f1-0885-4b33-bc7b-823e08e0a089 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:33.302175 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a66278f1-0885-4b33-bc7b-823e08e0a089 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1267 microversion: 2.1 time: 0.158999 Jul 11 14:42:33.302818 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 140/303] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:33 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1267 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:33.524346 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f3c160cb-ed24-49b8-9730-1bfdc5ea6210 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:33.524867 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f3c160cb-ed24-49b8-9730-1bfdc5ea6210 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:33.530986 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f3c160cb-ed24-49b8-9730-1bfdc5ea6210 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:33.531445 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f3c160cb-ed24-49b8-9730-1bfdc5ea6210 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:33.532218 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f3c160cb-ed24-49b8-9730-1bfdc5ea6210 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:33.532684 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f3c160cb-ed24-49b8-9730-1bfdc5ea6210 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:33.670062 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f3c160cb-ed24-49b8-9730-1bfdc5ea6210 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:33.678723 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f3c160cb-ed24-49b8-9730-1bfdc5ea6210 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1722 microversion: 2.1 time: 0.158910 Jul 11 14:42:33.679304 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 164/304] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:33 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1722 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:33.704406 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-34788888-65b8-407d-8f95-0d9b1a32a38d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:33.705013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-34788888-65b8-407d-8f95-0d9b1a32a38d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:33.712298 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-34788888-65b8-407d-8f95-0d9b1a32a38d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:33.712748 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-34788888-65b8-407d-8f95-0d9b1a32a38d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:33.713504 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-34788888-65b8-407d-8f95-0d9b1a32a38d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:33.713938 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-34788888-65b8-407d-8f95-0d9b1a32a38d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:33.833266 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-446c1f3f-4450-486c-9c41-4715fe8fb223 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:33.833899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-446c1f3f-4450-486c-9c41-4715fe8fb223 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:33.841103 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-446c1f3f-4450-486c-9c41-4715fe8fb223 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:33.841554 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-446c1f3f-4450-486c-9c41-4715fe8fb223 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:33.842319 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-446c1f3f-4450-486c-9c41-4715fe8fb223 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:33.842912 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-446c1f3f-4450-486c-9c41-4715fe8fb223 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:33.850278 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-34788888-65b8-407d-8f95-0d9b1a32a38d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:33.858111 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-34788888-65b8-407d-8f95-0d9b1a32a38d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.157454 Jul 11 14:42:33.858677 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 141/305] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:33 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:33.896475 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8d5081a3-0bc4-4206-b863-6cf6636fff15 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:33.897089 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-8d5081a3-0bc4-4206-b863-6cf6636fff15 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:33.903193 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d5081a3-0bc4-4206-b863-6cf6636fff15 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:33.903647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d5081a3-0bc4-4206-b863-6cf6636fff15 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:33.904420 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d5081a3-0bc4-4206-b863-6cf6636fff15 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:33.904877 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d5081a3-0bc4-4206-b863-6cf6636fff15 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:33.984335 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-446c1f3f-4450-486c-9c41-4715fe8fb223 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:33.994688 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-446c1f3f-4450-486c-9c41-4715fe8fb223 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:33.996219 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-446c1f3f-4450-486c-9c41-4715fe8fb223 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:33.997363 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-446c1f3f-4450-486c-9c41-4715fe8fb223 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.169569 Jul 11 14:42:33.997929 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 165/306] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:33 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:42:34.050398 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8d5081a3-0bc4-4206-b863-6cf6636fff15 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:34.058975 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8d5081a3-0bc4-4206-b863-6cf6636fff15 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1366 microversion: 2.1 time: 0.166211 Jul 11 14:42:34.060252 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 142/307] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:33 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1366 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:34.319514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8a536964-2d13-4f21-a979-51ff76807120 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:34.320029 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8a536964-2d13-4f21-a979-51ff76807120 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:34.322051 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:34.327492 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8a536964-2d13-4f21-a979-51ff76807120 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:34.327943 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8a536964-2d13-4f21-a979-51ff76807120 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:34.328708 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8a536964-2d13-4f21-a979-51ff76807120 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:34.329144 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8a536964-2d13-4f21-a979-51ff76807120 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:34.462801 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8a536964-2d13-4f21-a979-51ff76807120 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:34.470704 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8a536964-2d13-4f21-a979-51ff76807120 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1267 microversion: 2.1 time: 0.155140 Jul 11 14:42:34.471275 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 166/308] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:34 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1267 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:34.696021 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9508ee92-8f2c-49a3-9706-85b141c0ebcd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:34.696633 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-9508ee92-8f2c-49a3-9706-85b141c0ebcd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:34.702749 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9508ee92-8f2c-49a3-9706-85b141c0ebcd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:34.703219 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9508ee92-8f2c-49a3-9706-85b141c0ebcd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:34.704004 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9508ee92-8f2c-49a3-9706-85b141c0ebcd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:34.704438 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9508ee92-8f2c-49a3-9706-85b141c0ebcd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:34.875925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-10bf0717-df99-4585-a902-72624c1526b2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:34.876541 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-10bf0717-df99-4585-a902-72624c1526b2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:34.882828 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-10bf0717-df99-4585-a902-72624c1526b2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:34.883316 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-10bf0717-df99-4585-a902-72624c1526b2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:34.884093 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-10bf0717-df99-4585-a902-72624c1526b2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:34.884541 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-10bf0717-df99-4585-a902-72624c1526b2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:34.993573 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-9508ee92-8f2c-49a3-9706-85b141c0ebcd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:35.002556 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9508ee92-8f2c-49a3-9706-85b141c0ebcd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1722 microversion: 2.1 time: 0.310273 Jul 11 14:42:35.015562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 143/309] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:34 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1722 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:35.020462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7bad03d5-02cd-48da-a851-d4abd9c66c30 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:35.021078 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-7bad03d5-02cd-48da-a851-d4abd9c66c30 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:35.028076 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7bad03d5-02cd-48da-a851-d4abd9c66c30 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:35.028529 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7bad03d5-02cd-48da-a851-d4abd9c66c30 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:35.029307 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7bad03d5-02cd-48da-a851-d4abd9c66c30 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:35.029752 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7bad03d5-02cd-48da-a851-d4abd9c66c30 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:35.041664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-10bf0717-df99-4585-a902-72624c1526b2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:35.052520 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-10bf0717-df99-4585-a902-72624c1526b2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.179850 Jul 11 14:42:35.053831 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 167/310] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:34 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:35.077738 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-93928800-d03e-44a8-a97d-28fbd12a4ca4 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:35.078398 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-93928800-d03e-44a8-a97d-28fbd12a4ca4 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:35.085011 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-93928800-d03e-44a8-a97d-28fbd12a4ca4 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:35.085462 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-93928800-d03e-44a8-a97d-28fbd12a4ca4 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:35.086826 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-93928800-d03e-44a8-a97d-28fbd12a4ca4 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:35.087302 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-93928800-d03e-44a8-a97d-28fbd12a4ca4 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:35.168717 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7bad03d5-02cd-48da-a851-d4abd9c66c30 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:35.176887 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7bad03d5-02cd-48da-a851-d4abd9c66c30 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:35.178385 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7bad03d5-02cd-48da-a851-d4abd9c66c30 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:35.180208 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-7bad03d5-02cd-48da-a851-d4abd9c66c30 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.162954 Jul 11 14:42:35.180515 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 144/311] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:35 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:42:35.226301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-93928800-d03e-44a8-a97d-28fbd12a4ca4 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:35.233757 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-93928800-d03e-44a8-a97d-28fbd12a4ca4 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1366 microversion: 2.1 time: 0.161417 Jul 11 14:42:35.234312 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 168/312] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:35 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1366 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:35.487600 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-84a4f85c-db88-4f1e-809d-75d3562acc98 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:35.488213 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-84a4f85c-db88-4f1e-809d-75d3562acc98 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:35.494847 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-84a4f85c-db88-4f1e-809d-75d3562acc98 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:35.495299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-84a4f85c-db88-4f1e-809d-75d3562acc98 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:35.496060 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-84a4f85c-db88-4f1e-809d-75d3562acc98 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:35.496498 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-84a4f85c-db88-4f1e-809d-75d3562acc98 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:35.635912 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-84a4f85c-db88-4f1e-809d-75d3562acc98 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:35.643343 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-84a4f85c-db88-4f1e-809d-75d3562acc98 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1267 microversion: 2.1 time: 0.159606 Jul 11 14:42:35.643900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 145/313] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:35 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1267 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:36.020127 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-998b8a7f-cc99-4839-af38-7c366acda071 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:36.020664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-998b8a7f-cc99-4839-af38-7c366acda071 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:36.022544 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=4 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:36.023203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=12 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:36.028941 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-998b8a7f-cc99-4839-af38-7c366acda071 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:36.029391 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-998b8a7f-cc99-4839-af38-7c366acda071 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:36.030191 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-998b8a7f-cc99-4839-af38-7c366acda071 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:36.030663 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-998b8a7f-cc99-4839-af38-7c366acda071 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:36.070246 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-cc8bf012-a9f8-4bb8-be1f-ad00f70fd460 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:36.070944 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-cc8bf012-a9f8-4bb8-be1f-ad00f70fd460 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:36.077627 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc8bf012-a9f8-4bb8-be1f-ad00f70fd460 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:36.078078 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc8bf012-a9f8-4bb8-be1f-ad00f70fd460 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:36.078892 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc8bf012-a9f8-4bb8-be1f-ad00f70fd460 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:36.079318 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc8bf012-a9f8-4bb8-be1f-ad00f70fd460 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:36.161828 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-998b8a7f-cc99-4839-af38-7c366acda071 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:36.171117 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-998b8a7f-cc99-4839-af38-7c366acda071 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1722 microversion: 2.1 time: 0.155125 Jul 11 14:42:36.171691 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 169/314] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:36 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1722 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:36.197375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3fb62d18-71b0-4161-b69f-a22043d28ba1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:36.198000 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3fb62d18-71b0-4161-b69f-a22043d28ba1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:36.204763 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3fb62d18-71b0-4161-b69f-a22043d28ba1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:36.205220 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3fb62d18-71b0-4161-b69f-a22043d28ba1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:36.206356 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3fb62d18-71b0-4161-b69f-a22043d28ba1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:36.206888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3fb62d18-71b0-4161-b69f-a22043d28ba1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:36.220586 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-cc8bf012-a9f8-4bb8-be1f-ad00f70fd460 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:36.228607 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-cc8bf012-a9f8-4bb8-be1f-ad00f70fd460 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.162128 Jul 11 14:42:36.229171 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 146/315] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:36 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:36.250176 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-308fa18e-e91d-4a0d-b739-ef979e908d2b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:36.250847 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-308fa18e-e91d-4a0d-b739-ef979e908d2b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:36.256936 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-308fa18e-e91d-4a0d-b739-ef979e908d2b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:36.257353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-308fa18e-e91d-4a0d-b739-ef979e908d2b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:36.258389 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-308fa18e-e91d-4a0d-b739-ef979e908d2b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:36.258882 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-308fa18e-e91d-4a0d-b739-ef979e908d2b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:36.288986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=8 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:36.340513 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3fb62d18-71b0-4161-b69f-a22043d28ba1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:36.349758 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3fb62d18-71b0-4161-b69f-a22043d28ba1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:36.351319 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3fb62d18-71b0-4161-b69f-a22043d28ba1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:36.353199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3fb62d18-71b0-4161-b69f-a22043d28ba1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.159917 Jul 11 14:42:36.364372 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 170/316] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:36 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:42:36.394591 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-308fa18e-e91d-4a0d-b739-ef979e908d2b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:36.402683 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-308fa18e-e91d-4a0d-b739-ef979e908d2b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1366 microversion: 2.1 time: 0.156147 Jul 11 14:42:36.403243 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 147/317] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:36 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1366 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:36.660254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a34b0aa9-a065-4b20-8802-a7eb8a62f733 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:36.660855 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a34b0aa9-a065-4b20-8802-a7eb8a62f733 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:36.667397 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a34b0aa9-a065-4b20-8802-a7eb8a62f733 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:36.667868 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a34b0aa9-a065-4b20-8802-a7eb8a62f733 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:36.668691 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a34b0aa9-a065-4b20-8802-a7eb8a62f733 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:36.669136 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a34b0aa9-a065-4b20-8802-a7eb8a62f733 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:36.804753 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a34b0aa9-a065-4b20-8802-a7eb8a62f733 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:36.812791 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a34b0aa9-a065-4b20-8802-a7eb8a62f733 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1267 microversion: 2.1 time: 0.156437 Jul 11 14:42:36.813369 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 171/318] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:36 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1267 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:37.188374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-94a44515-1a84-4d78-b915-0940db123ad8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:37.189083 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-94a44515-1a84-4d78-b915-0940db123ad8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:37.196129 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-94a44515-1a84-4d78-b915-0940db123ad8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:37.196574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-94a44515-1a84-4d78-b915-0940db123ad8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:37.197331 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-94a44515-1a84-4d78-b915-0940db123ad8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:37.197775 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-94a44515-1a84-4d78-b915-0940db123ad8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:37.245955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a1449672-8d62-4290-b60c-dc948d59ac3d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:37.246697 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a1449672-8d62-4290-b60c-dc948d59ac3d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:37.253200 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a1449672-8d62-4290-b60c-dc948d59ac3d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:37.253776 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a1449672-8d62-4290-b60c-dc948d59ac3d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:37.254929 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a1449672-8d62-4290-b60c-dc948d59ac3d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:37.255580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a1449672-8d62-4290-b60c-dc948d59ac3d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:37.331221 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-94a44515-1a84-4d78-b915-0940db123ad8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:37.339291 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-94a44515-1a84-4d78-b915-0940db123ad8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1722 microversion: 2.1 time: 0.154934 Jul 11 14:42:37.339860 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 148/319] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:37 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1722 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:37.372435 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-87d5a0f0-400a-49b0-8a68-1b5177e6133e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:37.373028 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-87d5a0f0-400a-49b0-8a68-1b5177e6133e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:37.382202 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-87d5a0f0-400a-49b0-8a68-1b5177e6133e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:37.382202 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-87d5a0f0-400a-49b0-8a68-1b5177e6133e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:37.382202 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-87d5a0f0-400a-49b0-8a68-1b5177e6133e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:37.382202 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-87d5a0f0-400a-49b0-8a68-1b5177e6133e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:37.394927 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a1449672-8d62-4290-b60c-dc948d59ac3d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:37.404069 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a1449672-8d62-4290-b60c-dc948d59ac3d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.162617 Jul 11 14:42:37.405407 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 172/320] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:37 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:37.420481 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a66ade46-17bd-4405-891e-1d95dacf68e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:37.421088 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a66ade46-17bd-4405-891e-1d95dacf68e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:37.427245 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a66ade46-17bd-4405-891e-1d95dacf68e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:37.427748 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a66ade46-17bd-4405-891e-1d95dacf68e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:37.428507 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a66ade46-17bd-4405-891e-1d95dacf68e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:37.428940 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a66ade46-17bd-4405-891e-1d95dacf68e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:37.515401 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-87d5a0f0-400a-49b0-8a68-1b5177e6133e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:37.523581 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-87d5a0f0-400a-49b0-8a68-1b5177e6133e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:37.525100 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-87d5a0f0-400a-49b0-8a68-1b5177e6133e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:37.526263 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-87d5a0f0-400a-49b0-8a68-1b5177e6133e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.158503 Jul 11 14:42:37.526870 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 149/321] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:37 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:42:37.563713 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a66ade46-17bd-4405-891e-1d95dacf68e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:37.571091 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a66ade46-17bd-4405-891e-1d95dacf68e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1366 microversion: 2.1 time: 0.154472 Jul 11 14:42:37.571653 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 173/322] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:37 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:37.829351 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-0d8e114f-a3ba-45c4-97ad-10a03bc82078 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:37.829964 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-0d8e114f-a3ba-45c4-97ad-10a03bc82078 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:37.837052 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0d8e114f-a3ba-45c4-97ad-10a03bc82078 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:37.837500 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0d8e114f-a3ba-45c4-97ad-10a03bc82078 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:37.838259 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0d8e114f-a3ba-45c4-97ad-10a03bc82078 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:37.838726 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0d8e114f-a3ba-45c4-97ad-10a03bc82078 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:37.978200 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-0d8e114f-a3ba-45c4-97ad-10a03bc82078 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:37.985584 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-0d8e114f-a3ba-45c4-97ad-10a03bc82078 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1267 microversion: 2.1 time: 0.160125 Jul 11 14:42:37.986132 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 150/323] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:37 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1267 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:38.356352 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-63a82733-8632-46dd-a41c-516eb1c54023 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:38.356976 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-63a82733-8632-46dd-a41c-516eb1c54023 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:38.363125 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-63a82733-8632-46dd-a41c-516eb1c54023 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:38.363582 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-63a82733-8632-46dd-a41c-516eb1c54023 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:38.364350 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-63a82733-8632-46dd-a41c-516eb1c54023 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:38.364794 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-63a82733-8632-46dd-a41c-516eb1c54023 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:38.421780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-06544eb6-d192-4205-af92-7651e044b17a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:38.422512 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-06544eb6-d192-4205-af92-7651e044b17a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:38.439044 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-06544eb6-d192-4205-af92-7651e044b17a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:38.439652 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-06544eb6-d192-4205-af92-7651e044b17a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:38.440547 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-06544eb6-d192-4205-af92-7651e044b17a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:38.441098 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-06544eb6-d192-4205-af92-7651e044b17a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:38.458253 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=100 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:38.521457 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-63a82733-8632-46dd-a41c-516eb1c54023 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:38.529573 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-63a82733-8632-46dd-a41c-516eb1c54023 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1722 microversion: 2.1 time: 0.177428 Jul 11 14:42:38.530133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 174/324] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:38 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1722 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:38.549251 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-55bd2cad-429a-4353-b85c-30b58cf164bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:38.550505 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-55bd2cad-429a-4353-b85c-30b58cf164bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:38.556627 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-55bd2cad-429a-4353-b85c-30b58cf164bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:38.557063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-55bd2cad-429a-4353-b85c-30b58cf164bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:38.557845 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-55bd2cad-429a-4353-b85c-30b58cf164bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:38.558246 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-55bd2cad-429a-4353-b85c-30b58cf164bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:38.574839 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-06544eb6-d192-4205-af92-7651e044b17a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:38.582652 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-06544eb6-d192-4205-af92-7651e044b17a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.164978 Jul 11 14:42:38.583429 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 151/325] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:38 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:38.587795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d37ab648-2b38-48ca-a97e-2836301356c1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:38.588402 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d37ab648-2b38-48ca-a97e-2836301356c1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:38.594382 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d37ab648-2b38-48ca-a97e-2836301356c1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:38.594863 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d37ab648-2b38-48ca-a97e-2836301356c1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:38.595628 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d37ab648-2b38-48ca-a97e-2836301356c1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:38.596065 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d37ab648-2b38-48ca-a97e-2836301356c1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:38.708247 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-55bd2cad-429a-4353-b85c-30b58cf164bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:38.716536 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-55bd2cad-429a-4353-b85c-30b58cf164bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:38.718062 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-55bd2cad-429a-4353-b85c-30b58cf164bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:38.719253 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-55bd2cad-429a-4353-b85c-30b58cf164bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.173917 Jul 11 14:42:38.719817 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 175/326] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:38 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:42:38.752108 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d37ab648-2b38-48ca-a97e-2836301356c1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:38.759464 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d37ab648-2b38-48ca-a97e-2836301356c1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1366 microversion: 2.1 time: 0.175247 Jul 11 14:42:38.760025 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 152/327] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:38 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1366 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:39.002031 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2cd55064-3a6e-4bc3-b61c-a756af5538f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:39.002714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2cd55064-3a6e-4bc3-b61c-a756af5538f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:39.009755 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2cd55064-3a6e-4bc3-b61c-a756af5538f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:39.010203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2cd55064-3a6e-4bc3-b61c-a756af5538f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:39.011010 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2cd55064-3a6e-4bc3-b61c-a756af5538f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:39.011447 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2cd55064-3a6e-4bc3-b61c-a756af5538f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:39.425570 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-2cd55064-3a6e-4bc3-b61c-a756af5538f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:39.433069 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2cd55064-3a6e-4bc3-b61c-a756af5538f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1267 microversion: 2.1 time: 0.434910 Jul 11 14:42:39.433635 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 176/328] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:38 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1267 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:39.548699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d5d1d3a6-6189-44b4-a339-c37f7b1c8c4a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:39.549488 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d5d1d3a6-6189-44b4-a339-c37f7b1c8c4a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:39.556167 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d5d1d3a6-6189-44b4-a339-c37f7b1c8c4a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:39.556649 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d5d1d3a6-6189-44b4-a339-c37f7b1c8c4a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:39.557414 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d5d1d3a6-6189-44b4-a339-c37f7b1c8c4a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:39.557855 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d5d1d3a6-6189-44b4-a339-c37f7b1c8c4a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:39.599528 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a06ff21f-2dd1-41c9-b515-c7eb34864a12 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:39.600173 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a06ff21f-2dd1-41c9-b515-c7eb34864a12 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:39.606400 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a06ff21f-2dd1-41c9-b515-c7eb34864a12 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:39.606886 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a06ff21f-2dd1-41c9-b515-c7eb34864a12 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:39.607655 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a06ff21f-2dd1-41c9-b515-c7eb34864a12 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:39.608089 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a06ff21f-2dd1-41c9-b515-c7eb34864a12 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:39.700961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d5d1d3a6-6189-44b4-a339-c37f7b1c8c4a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:39.709996 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d5d1d3a6-6189-44b4-a339-c37f7b1c8c4a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1722 microversion: 2.1 time: 0.165387 Jul 11 14:42:39.710597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 153/329] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:39 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1722 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:39.739559 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d171d201-58e4-4840-a228-dc7d4c076d79 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:39.739901 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d171d201-58e4-4840-a228-dc7d4c076d79 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:39.746251 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=99,nova_api:INSERT=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:39.747907 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d171d201-58e4-4840-a228-dc7d4c076d79 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:39.748397 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d171d201-58e4-4840-a228-dc7d4c076d79 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:39.749166 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d171d201-58e4-4840-a228-dc7d4c076d79 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:39.749908 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d171d201-58e4-4840-a228-dc7d4c076d79 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:39.756146 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a06ff21f-2dd1-41c9-b515-c7eb34864a12 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:39.760714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=100 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:39.766318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a06ff21f-2dd1-41c9-b515-c7eb34864a12 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.170376 Jul 11 14:42:39.767157 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 177/330] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:39 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:39.776721 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-af2a151d-30e0-4143-8970-240010dd1595 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:39.777282 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-af2a151d-30e0-4143-8970-240010dd1595 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:39.783378 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-af2a151d-30e0-4143-8970-240010dd1595 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:39.783825 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-af2a151d-30e0-4143-8970-240010dd1595 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:39.784593 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-af2a151d-30e0-4143-8970-240010dd1595 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:39.785035 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-af2a151d-30e0-4143-8970-240010dd1595 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:39.899243 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d171d201-58e4-4840-a228-dc7d4c076d79 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:39.908234 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d171d201-58e4-4840-a228-dc7d4c076d79 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:39.910037 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d171d201-58e4-4840-a228-dc7d4c076d79 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:39.911304 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d171d201-58e4-4840-a228-dc7d4c076d79 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.176967 Jul 11 14:42:39.912198 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 154/331] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:39 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:42:39.939357 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-af2a151d-30e0-4143-8970-240010dd1595 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:39.947815 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-af2a151d-30e0-4143-8970-240010dd1595 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1366 microversion: 2.1 time: 0.175468 Jul 11 14:42:39.948787 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 178/332] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:39 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1366 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:40.004163 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-2a5a181f-80ce-4366-a509-ceda3fee143e req-18474349-df69-460e-bfda-27cccc4cca91 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3a91aed4-52ff-4220-bc8f-571c50452865", "name": "network-vif-plugged", "status": "completed", "tag": "957d5045-a8c1-4d28-8ee4-755331ae2c84"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:40.012922 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-2a5a181f-80ce-4366-a509-ceda3fee143e req-18474349-df69-460e-bfda-27cccc4cca91 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:40.013373 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-2a5a181f-80ce-4366-a509-ceda3fee143e req-18474349-df69-460e-bfda-27cccc4cca91 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:40.025621 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-2a5a181f-80ce-4366-a509-ceda3fee143e req-18474349-df69-460e-bfda-27cccc4cca91 service nova] Creating event network-vif-plugged:957d5045-a8c1-4d28-8ee4-755331ae2c84 for instance 3a91aed4-52ff-4220-bc8f-571c50452865 on hv-cinder-maste Jul 11 14:42:40.032209 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-2a5a181f-80ce-4366-a509-ceda3fee143e req-18474349-df69-460e-bfda-27cccc4cca91 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031829 Jul 11 14:42:40.032972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 155/333] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:42:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:42:40.451172 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3a4482bb-89f6-44ae-8907-9ec7e3c0b497 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:40.451795 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3a4482bb-89f6-44ae-8907-9ec7e3c0b497 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:40.458649 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3a4482bb-89f6-44ae-8907-9ec7e3c0b497 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:40.459114 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3a4482bb-89f6-44ae-8907-9ec7e3c0b497 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:40.459872 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3a4482bb-89f6-44ae-8907-9ec7e3c0b497 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:40.460306 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3a4482bb-89f6-44ae-8907-9ec7e3c0b497 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:40.599109 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3a4482bb-89f6-44ae-8907-9ec7e3c0b497 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:40.609059 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3a4482bb-89f6-44ae-8907-9ec7e3c0b497 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1267 microversion: 2.1 time: 0.161734 Jul 11 14:42:40.609931 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 179/334] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:40 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1267 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:40.729269 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3f4757d4-6b96-4098-a87c-f2f65f6b50c1 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:40.730091 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-3f4757d4-6b96-4098-a87c-f2f65f6b50c1 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:40.736900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3f4757d4-6b96-4098-a87c-f2f65f6b50c1 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:40.737433 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3f4757d4-6b96-4098-a87c-f2f65f6b50c1 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:40.738219 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3f4757d4-6b96-4098-a87c-f2f65f6b50c1 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:40.738694 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3f4757d4-6b96-4098-a87c-f2f65f6b50c1 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:40.783771 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-6da977b3-fa9a-47b0-8d60-1824462365b4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:40.784397 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-6da977b3-fa9a-47b0-8d60-1824462365b4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:40.792875 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6da977b3-fa9a-47b0-8d60-1824462365b4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:40.793421 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6da977b3-fa9a-47b0-8d60-1824462365b4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:40.794199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6da977b3-fa9a-47b0-8d60-1824462365b4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:40.794678 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6da977b3-fa9a-47b0-8d60-1824462365b4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:40.878734 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-3f4757d4-6b96-4098-a87c-f2f65f6b50c1 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:40.886916 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-3f4757d4-6b96-4098-a87c-f2f65f6b50c1 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1722 microversion: 2.1 time: 0.162346 Jul 11 14:42:40.887771 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 156/335] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:40 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1722 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:40.930646 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f952415e-c9d8-4d79-b4ca-bd48b1d4e847 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:40.931289 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f952415e-c9d8-4d79-b4ca-bd48b1d4e847 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:40.932506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-6da977b3-fa9a-47b0-8d60-1824462365b4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:40.937676 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f952415e-c9d8-4d79-b4ca-bd48b1d4e847 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:40.938127 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f952415e-c9d8-4d79-b4ca-bd48b1d4e847 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:40.938967 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f952415e-c9d8-4d79-b4ca-bd48b1d4e847 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:40.939388 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f952415e-c9d8-4d79-b4ca-bd48b1d4e847 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:40.940420 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-6da977b3-fa9a-47b0-8d60-1824462365b4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.160486 Jul 11 14:42:40.940990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 180/336] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:40 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:40.964773 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1e478500-af24-4d43-b189-1f65b1ac9ee3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:40.965535 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1e478500-af24-4d43-b189-1f65b1ac9ee3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:40.971778 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1e478500-af24-4d43-b189-1f65b1ac9ee3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:40.972407 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1e478500-af24-4d43-b189-1f65b1ac9ee3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:40.973154 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1e478500-af24-4d43-b189-1f65b1ac9ee3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:40.973666 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1e478500-af24-4d43-b189-1f65b1ac9ee3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:41.074053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f952415e-c9d8-4d79-b4ca-bd48b1d4e847 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:41.082340 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f952415e-c9d8-4d79-b4ca-bd48b1d4e847 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:41.083863 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f952415e-c9d8-4d79-b4ca-bd48b1d4e847 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:41.085006 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f952415e-c9d8-4d79-b4ca-bd48b1d4e847 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.158931 Jul 11 14:42:41.085561 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 157/337] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:40 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:42:41.105001 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1e478500-af24-4d43-b189-1f65b1ac9ee3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:41.112502 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1e478500-af24-4d43-b189-1f65b1ac9ee3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1366 microversion: 2.1 time: 0.151594 Jul 11 14:42:41.113104 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 181/338] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:40 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1366 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:41.627225 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e143b111-4b0c-4240-a9dd-c7555681fe9d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:41.627845 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-e143b111-4b0c-4240-a9dd-c7555681fe9d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:41.634147 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e143b111-4b0c-4240-a9dd-c7555681fe9d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:41.634698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e143b111-4b0c-4240-a9dd-c7555681fe9d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:41.635468 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e143b111-4b0c-4240-a9dd-c7555681fe9d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:41.635906 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e143b111-4b0c-4240-a9dd-c7555681fe9d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:41.770114 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e143b111-4b0c-4240-a9dd-c7555681fe9d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:41.778504 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e143b111-4b0c-4240-a9dd-c7555681fe9d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1267 microversion: 2.1 time: 0.155334 Jul 11 14:42:41.779171 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 158/339] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:41 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1267 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:41.847030 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-87de3ba2-4a37-465b-b250-dacd16c85052 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "fee0dc64-49de-4fe5-8b94-881d049f6d4b", "device": "/dev/sdb"}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:41.848708 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-87de3ba2-4a37-465b-b250-dacd16c85052 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:41.854924 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-87de3ba2-4a37-465b-b250-dacd16c85052 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:41.855390 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-87de3ba2-4a37-465b-b250-dacd16c85052 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:41.856194 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-87de3ba2-4a37-465b-b250-dacd16c85052 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:41.856633 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-87de3ba2-4a37-465b-b250-dacd16c85052 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:41.912214 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-82f3cd69-39fc-46db-8e5e-2f3f7c496f1a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:41.912987 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-82f3cd69-39fc-46db-8e5e-2f3f7c496f1a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:41.932843 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-82f3cd69-39fc-46db-8e5e-2f3f7c496f1a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:41.934369 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-82f3cd69-39fc-46db-8e5e-2f3f7c496f1a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:41.935877 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-82f3cd69-39fc-46db-8e5e-2f3f7c496f1a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:41.936558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-82f3cd69-39fc-46db-8e5e-2f3f7c496f1a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:42.078529 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-82f3cd69-39fc-46db-8e5e-2f3f7c496f1a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:42.086748 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-82f3cd69-39fc-46db-8e5e-2f3f7c496f1a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1722 microversion: 2.1 time: 0.178376 Jul 11 14:42:42.087306 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 159/340] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:41 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1722 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:42.091739 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-821fc492-f5ea-4a51-aff6-e5e01007b9c9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:42.092334 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-821fc492-f5ea-4a51-aff6-e5e01007b9c9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:42.098619 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-821fc492-f5ea-4a51-aff6-e5e01007b9c9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:42.099068 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-821fc492-f5ea-4a51-aff6-e5e01007b9c9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:42.099828 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-821fc492-f5ea-4a51-aff6-e5e01007b9c9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:42.100264 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-821fc492-f5ea-4a51-aff6-e5e01007b9c9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:42.232303 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-821fc492-f5ea-4a51-aff6-e5e01007b9c9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:42.239799 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-821fc492-f5ea-4a51-aff6-e5e01007b9c9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1501 microversion: 2.1 time: 0.151557 Jul 11 14:42:42.240355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 160/341] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:42 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1501 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:42.244633 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-d7df07f1-0efe-453e-9f6b-58e737930aac req-6c2fcbd3-5cd8-4699-a847-43afb2611015 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7a49d1ca-7491-41e0-b0d8-37c92b9d3562", "name": "network-vif-plugged", "status": "completed", "tag": "c62c0fe9-fd78-4a72-aef8-9bd42e6d81de"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:42.252364 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-d7df07f1-0efe-453e-9f6b-58e737930aac req-6c2fcbd3-5cd8-4699-a847-43afb2611015 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:42.252804 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-d7df07f1-0efe-453e-9f6b-58e737930aac req-6c2fcbd3-5cd8-4699-a847-43afb2611015 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:42.263677 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-d7df07f1-0efe-453e-9f6b-58e737930aac req-6c2fcbd3-5cd8-4699-a847-43afb2611015 service nova] Creating event network-vif-plugged:c62c0fe9-fd78-4a72-aef8-9bd42e6d81de for instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 on hv-cinder-maste Jul 11 14:42:42.268391 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-d7df07f1-0efe-453e-9f6b-58e737930aac req-6c2fcbd3-5cd8-4699-a847-43afb2611015 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.026954 Jul 11 14:42:42.268958 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 161/342] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:42:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:42:42.274361 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8f965f0f-67c9-4c5d-9a41-5fa436e7134e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:42.275081 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-8f965f0f-67c9-4c5d-9a41-5fa436e7134e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:42.280990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8f965f0f-67c9-4c5d-9a41-5fa436e7134e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:42.281436 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8f965f0f-67c9-4c5d-9a41-5fa436e7134e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:42.282213 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8f965f0f-67c9-4c5d-9a41-5fa436e7134e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:42.282675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8f965f0f-67c9-4c5d-9a41-5fa436e7134e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:42.418133 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8f965f0f-67c9-4c5d-9a41-5fa436e7134e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:42.426211 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8f965f0f-67c9-4c5d-9a41-5fa436e7134e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:42.427798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8f965f0f-67c9-4c5d-9a41-5fa436e7134e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:42.428950 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8f965f0f-67c9-4c5d-9a41-5fa436e7134e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.158952 Jul 11 14:42:42.429500 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 162/343] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:42 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:42:42.434707 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-61b6b3c2-830b-4320-a027-13f1adbd9922 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:42.435309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-61b6b3c2-830b-4320-a027-13f1adbd9922 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:42.450957 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-61b6b3c2-830b-4320-a027-13f1adbd9922 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:42.451461 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-61b6b3c2-830b-4320-a027-13f1adbd9922 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:42.452378 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-61b6b3c2-830b-4320-a027-13f1adbd9922 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:42.452890 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-61b6b3c2-830b-4320-a027-13f1adbd9922 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:42.579185 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-87de3ba2-4a37-465b-b250-dacd16c85052 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "POST /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.735875 Jul 11 14:42:42.585407 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 182/344] 10.222.0.72 () {60 vars in 1322 bytes} [Mon Jul 11 14:42:41 2022] POST /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c/os-volume_attachments => generated 194 bytes in 738 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:42:42.585722 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-510b4fbe-d868-4658-adda-6eaab3341150 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "996c27f4-1a7f-451d-970e-3b4f39bec2d8"}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:42.587141 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-510b4fbe-d868-4658-adda-6eaab3341150 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:42.603405 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-510b4fbe-d868-4658-adda-6eaab3341150 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:42.603871 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-510b4fbe-d868-4658-adda-6eaab3341150 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:42.604657 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-510b4fbe-d868-4658-adda-6eaab3341150 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:42.605102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-510b4fbe-d868-4658-adda-6eaab3341150 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:42.621127 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-61b6b3c2-830b-4320-a027-13f1adbd9922 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:42.630230 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-61b6b3c2-830b-4320-a027-13f1adbd9922 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1366 microversion: 2.1 time: 0.199190 Jul 11 14:42:42.633043 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-510b4fbe-d868-4658-adda-6eaab3341150 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] HTTP exception thrown: Cannot 'attach_volume' instance a0498735-9e0a-4f14-990d-a6d5076e742a while it is in vm_state rescued Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-510b4fbe-d868-4658-adda-6eaab3341150 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Request method failure captured: Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: POST /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a/os-volume_attachments HTTP/1.1 Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 74 Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZVO0vcEhdpPwwfJZnu1y3p7Tg0Q8GENrUVh0NohR9oAPNIjKFLY8v62KxMOiNHGfwF-EVGge1hPwlyTCX1uExRac1NToAISnaSDmVmpVWA-TKGS1rzgSZVkfiPxYTNCyZmEJBrMLK9NH_pMk0kTrm35o7KBc7uMvJqd_pTjsh0TKE Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: 98966d44ea88477b94215761cdc92ea7 Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1829929608 Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:42:42.634718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:42:42.636484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/98966d44ea88477b94215761cdc92ea7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/98966d44ea88477b94215761cdc92ea7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_98966d44ea88477b94215761cdc92ea7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/98966d44ea88477b94215761cdc92ea7"}]}] Jul 11 14:42:42.636484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1829929608 Jul 11 14:42:42.636484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: 98966d44ea88477b94215761cdc92ea7 Jul 11 14:42:42.636484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1829929608 Jul 11 14:42:42.636484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-ServerRescueNegativeTestJSON-1829929608-project Jul 11 14:42:42.636484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:42:42.636484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:42:42.636484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: fa478fb4b41e49c5b1f9291f925cca23 Jul 11 14:42:42.636484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1829929608-project Jul 11 14:42:42.636484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Jul 11 14:42:42.636484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {"volumeAttachment": {"volumeId": "996c27f4-1a7f-451d-970e-3b4f39bec2d8"}} Jul 11 14:42:42.636484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Cannot 'attach_volume' instance a0498735-9e0a-4f14-990d-a6d5076e742a while it is in vm_state rescued Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'server_id': 'a0498735-9e0a-4f14-990d-a6d5076e742a', 'body': {'volumeAttachment': {'volumeId': '996c27f4-1a7f-451d-970e-3b4f39bec2d8'}}} Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance a0498735-9e0a-4f14-990d-a6d5076e742a in vm_state rescued. Cannot attach_volume while the instance is in this state. Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:42:42.637553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance a0498735-9e0a-4f14-990d-a6d5076e742a while it is in vm_state rescued Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:42:42.638955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:42:42.640885 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:42:42.640885 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:42:42.640885 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance a0498735-9e0a-4f14-990d-a6d5076e742a while it is in vm_state rescued Jul 11 14:42:42.640885 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:42:42.640885 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-510b4fbe-d868-4658-adda-6eaab3341150 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Returning 409 to user: Cannot 'attach_volume' instance a0498735-9e0a-4f14-990d-a6d5076e742a while it is in vm_state rescued {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:42:42.640885 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-510b4fbe-d868-4658-adda-6eaab3341150 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "POST /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.054602 Jul 11 14:42:42.640885 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 183/345] 10.222.0.72 () {60 vars in 1322 bytes} [Mon Jul 11 14:42:42 2022] POST /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a/os-volume_attachments => generated 152 bytes in 56 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Jul 11 14:42:42.641567 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 163/346] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:42 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1366 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:42.645052 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-aa9b9077-e7c9-4147-909e-be5b165f90d0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:42.645585 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-aa9b9077-e7c9-4147-909e-be5b165f90d0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:42.651700 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-aa9b9077-e7c9-4147-909e-be5b165f90d0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:42.652157 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-aa9b9077-e7c9-4147-909e-be5b165f90d0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:42.652918 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-aa9b9077-e7c9-4147-909e-be5b165f90d0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:42.653360 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-aa9b9077-e7c9-4147-909e-be5b165f90d0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:42.718771 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-aa9b9077-e7c9-4147-909e-be5b165f90d0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "POST /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a/action" status: 202 len: 0 microversion: 2.1 time: 0.077328 Jul 11 14:42:42.719386 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 184/347] 10.222.0.72 () {60 vars in 1277 bytes} [Mon Jul 11 14:42:42 2022] POST /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a/action => generated 0 bytes in 79 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:42:42.728275 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-6b6682a8-9c3e-488f-aef6-4835e3ada5c0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:42.728882 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-6b6682a8-9c3e-488f-aef6-4835e3ada5c0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:42.735573 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6b6682a8-9c3e-488f-aef6-4835e3ada5c0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:42.736026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6b6682a8-9c3e-488f-aef6-4835e3ada5c0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:42.736783 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6b6682a8-9c3e-488f-aef6-4835e3ada5c0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:42.737219 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6b6682a8-9c3e-488f-aef6-4835e3ada5c0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:42.797187 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-43072bdf-ac8f-471e-8763-bf6228320c34 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:42.798212 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-43072bdf-ac8f-471e-8763-bf6228320c34 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:42.806408 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-43072bdf-ac8f-471e-8763-bf6228320c34 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:42.807157 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-43072bdf-ac8f-471e-8763-bf6228320c34 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:42.808087 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-43072bdf-ac8f-471e-8763-bf6228320c34 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:42.809588 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-43072bdf-ac8f-471e-8763-bf6228320c34 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:42.907461 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6b6682a8-9c3e-488f-aef6-4835e3ada5c0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:42.916046 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-6b6682a8-9c3e-488f-aef6-4835e3ada5c0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1509 microversion: 2.1 time: 0.191971 Jul 11 14:42:42.916730 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 164/348] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:42 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1509 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:42.969574 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-43072bdf-ac8f-471e-8763-bf6228320c34 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:42.977658 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-43072bdf-ac8f-471e-8763-bf6228320c34 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1267 microversion: 2.1 time: 0.185612 Jul 11 14:42:42.978215 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 185/349] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:42 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1267 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:43.104535 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-96bff32d-8605-4ea3-aef1-6eb569837ebc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:43.105078 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-96bff32d-8605-4ea3-aef1-6eb569837ebc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:43.111102 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-96bff32d-8605-4ea3-aef1-6eb569837ebc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:43.111550 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-96bff32d-8605-4ea3-aef1-6eb569837ebc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:43.112310 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-96bff32d-8605-4ea3-aef1-6eb569837ebc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:43.112750 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-96bff32d-8605-4ea3-aef1-6eb569837ebc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:43.240332 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-96bff32d-8605-4ea3-aef1-6eb569837ebc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:43.248456 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-96bff32d-8605-4ea3-aef1-6eb569837ebc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1722 microversion: 2.1 time: 0.148493 Jul 11 14:42:43.248981 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 165/350] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:43 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1722 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:43.447561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-688a3af7-2ae3-4c41-9610-14ec714ca7cf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:43.448181 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-688a3af7-2ae3-4c41-9610-14ec714ca7cf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:43.454656 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-688a3af7-2ae3-4c41-9610-14ec714ca7cf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:43.455111 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-688a3af7-2ae3-4c41-9610-14ec714ca7cf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:43.456109 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-688a3af7-2ae3-4c41-9610-14ec714ca7cf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:43.456550 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-688a3af7-2ae3-4c41-9610-14ec714ca7cf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:43.584724 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-688a3af7-2ae3-4c41-9610-14ec714ca7cf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:43.593365 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-688a3af7-2ae3-4c41-9610-14ec714ca7cf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:43.594929 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-688a3af7-2ae3-4c41-9610-14ec714ca7cf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:43.596058 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-688a3af7-2ae3-4c41-9610-14ec714ca7cf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.152863 Jul 11 14:42:43.596603 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 186/351] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:43 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:42:43.647106 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-823ab468-b750-43a4-b9e5-ae0a0d6f7b9c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:43.647637 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-823ab468-b750-43a4-b9e5-ae0a0d6f7b9c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:43.653719 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-823ab468-b750-43a4-b9e5-ae0a0d6f7b9c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:43.654197 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-823ab468-b750-43a4-b9e5-ae0a0d6f7b9c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:43.655026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-823ab468-b750-43a4-b9e5-ae0a0d6f7b9c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:43.655465 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-823ab468-b750-43a4-b9e5-ae0a0d6f7b9c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:43.806309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-823ab468-b750-43a4-b9e5-ae0a0d6f7b9c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:43.813713 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-823ab468-b750-43a4-b9e5-ae0a0d6f7b9c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1366 microversion: 2.1 time: 0.170497 Jul 11 14:42:43.814277 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 166/352] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:43 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1366 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:43.933667 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c63d9f15-2d16-4d21-bdc9-d60832cd02b3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:43.934201 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c63d9f15-2d16-4d21-bdc9-d60832cd02b3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:43.940256 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c63d9f15-2d16-4d21-bdc9-d60832cd02b3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:43.940713 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c63d9f15-2d16-4d21-bdc9-d60832cd02b3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:43.941473 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c63d9f15-2d16-4d21-bdc9-d60832cd02b3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:43.941942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c63d9f15-2d16-4d21-bdc9-d60832cd02b3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:43.993902 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-596a28d7-ac46-473c-a055-f641dd105de5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:43.994563 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-596a28d7-ac46-473c-a055-f641dd105de5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:44.000983 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-596a28d7-ac46-473c-a055-f641dd105de5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:44.001520 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-596a28d7-ac46-473c-a055-f641dd105de5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:44.002364 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-596a28d7-ac46-473c-a055-f641dd105de5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:44.002906 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-596a28d7-ac46-473c-a055-f641dd105de5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:44.077268 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c63d9f15-2d16-4d21-bdc9-d60832cd02b3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:44.085242 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c63d9f15-2d16-4d21-bdc9-d60832cd02b3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1509 microversion: 2.1 time: 0.155913 Jul 11 14:42:44.085915 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 187/353] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:43 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1509 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:44.141986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-596a28d7-ac46-473c-a055-f641dd105de5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:44.149761 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-596a28d7-ac46-473c-a055-f641dd105de5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1453 microversion: 2.1 time: 0.160427 Jul 11 14:42:44.150563 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 167/354] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:43 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1453 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:44.166577 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-46bd8ddc-8e2a-450a-8ab2-cf6dc222ec6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:44.167196 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-46bd8ddc-8e2a-450a-8ab2-cf6dc222ec6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:44.173891 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-46bd8ddc-8e2a-450a-8ab2-cf6dc222ec6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:44.174342 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-46bd8ddc-8e2a-450a-8ab2-cf6dc222ec6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:44.175136 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-46bd8ddc-8e2a-450a-8ab2-cf6dc222ec6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:44.175574 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-46bd8ddc-8e2a-450a-8ab2-cf6dc222ec6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:44.264825 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9de65cbc-d26f-4c9b-b888-300b900a4219 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:44.265435 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-9de65cbc-d26f-4c9b-b888-300b900a4219 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:44.271354 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9de65cbc-d26f-4c9b-b888-300b900a4219 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:44.271806 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9de65cbc-d26f-4c9b-b888-300b900a4219 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:44.272567 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9de65cbc-d26f-4c9b-b888-300b900a4219 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:44.273004 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9de65cbc-d26f-4c9b-b888-300b900a4219 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:44.307899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-46bd8ddc-8e2a-450a-8ab2-cf6dc222ec6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:44.315369 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-46bd8ddc-8e2a-450a-8ab2-cf6dc222ec6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1453 microversion: 2.1 time: 0.153154 Jul 11 14:42:44.315932 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 188/355] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:44 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1453 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:44.407889 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-9de65cbc-d26f-4c9b-b888-300b900a4219 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:44.416908 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9de65cbc-d26f-4c9b-b888-300b900a4219 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1722 microversion: 2.1 time: 0.155854 Jul 11 14:42:44.417475 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 168/356] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:44 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1722 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:44.616050 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2f733cf4-2d00-4e36-95cf-6fc105a233c7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:44.616769 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2f733cf4-2d00-4e36-95cf-6fc105a233c7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:44.625161 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2f733cf4-2d00-4e36-95cf-6fc105a233c7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:44.625612 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2f733cf4-2d00-4e36-95cf-6fc105a233c7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:44.626379 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2f733cf4-2d00-4e36-95cf-6fc105a233c7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:44.626889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2f733cf4-2d00-4e36-95cf-6fc105a233c7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:44.832357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-b3a30183-fd73-4b83-b31d-4366aff308c5 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:44.833122 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-b3a30183-fd73-4b83-b31d-4366aff308c5 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:44.840136 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b3a30183-fd73-4b83-b31d-4366aff308c5 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:44.840674 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b3a30183-fd73-4b83-b31d-4366aff308c5 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:44.841689 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b3a30183-fd73-4b83-b31d-4366aff308c5 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:44.842371 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b3a30183-fd73-4b83-b31d-4366aff308c5 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:44.858360 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=100 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:44.893896 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-2f733cf4-2d00-4e36-95cf-6fc105a233c7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:44.906048 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-2f733cf4-2d00-4e36-95cf-6fc105a233c7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:44.908276 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-2f733cf4-2d00-4e36-95cf-6fc105a233c7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:44.909974 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2f733cf4-2d00-4e36-95cf-6fc105a233c7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.299464 Jul 11 14:42:44.910838 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 189/357] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:44 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:42:44.987776 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-b3a30183-fd73-4b83-b31d-4366aff308c5 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:44.996552 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-b3a30183-fd73-4b83-b31d-4366aff308c5 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1366 microversion: 2.1 time: 0.168652 Jul 11 14:42:44.997400 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 169/358] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:44 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1366 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:45.102933 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-261551dc-1ed9-4437-95b7-96354a42a097 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:45.104160 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-261551dc-1ed9-4437-95b7-96354a42a097 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:45.112967 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-261551dc-1ed9-4437-95b7-96354a42a097 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:45.114179 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-261551dc-1ed9-4437-95b7-96354a42a097 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:45.115336 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-261551dc-1ed9-4437-95b7-96354a42a097 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:45.115964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-261551dc-1ed9-4437-95b7-96354a42a097 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:45.401894 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-261551dc-1ed9-4437-95b7-96354a42a097 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:45.411226 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-261551dc-1ed9-4437-95b7-96354a42a097 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1509 microversion: 2.1 time: 0.312138 Jul 11 14:42:45.411813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 190/359] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:45 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1509 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:45.435266 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c96df859-f8ff-42eb-a1e2-e4fb4eb659c4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:45.436206 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c96df859-f8ff-42eb-a1e2-e4fb4eb659c4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:45.444020 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c96df859-f8ff-42eb-a1e2-e4fb4eb659c4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:45.444571 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c96df859-f8ff-42eb-a1e2-e4fb4eb659c4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:45.446361 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c96df859-f8ff-42eb-a1e2-e4fb4eb659c4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:45.446996 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c96df859-f8ff-42eb-a1e2-e4fb4eb659c4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:45.602797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c96df859-f8ff-42eb-a1e2-e4fb4eb659c4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:45.610825 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c96df859-f8ff-42eb-a1e2-e4fb4eb659c4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1699 microversion: 2.1 time: 0.181230 Jul 11 14:42:45.611624 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 170/360] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:45 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1699 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:45.627769 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3267bc7b-cbcd-41af-93a6-38f7e98b5599 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:45.635391 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3267bc7b-cbcd-41af-93a6-38f7e98b5599 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:45.635897 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3267bc7b-cbcd-41af-93a6-38f7e98b5599 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:45.658684 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3267bc7b-cbcd-41af-93a6-38f7e98b5599 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Going to try to start instance {{(pid=130159) start /opt/stack/nova/nova/compute/api.py:2730}} Jul 11 14:42:45.688595 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:42:45.706798 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3267bc7b-cbcd-41af-93a6-38f7e98b5599 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "POST /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d/action" status: 202 len: 0 microversion: 2.1 time: 0.082507 Jul 11 14:42:45.707389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 191/361] 10.222.0.72 () {60 vars in 1277 bytes} [Mon Jul 11 14:42:45 2022] POST /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d/action => generated 0 bytes in 84 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:42:45.715790 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7f7f6f3e-239c-48bd-8b4d-52158f1e5df2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:45.716490 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-7f7f6f3e-239c-48bd-8b4d-52158f1e5df2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:45.722664 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7f7f6f3e-239c-48bd-8b4d-52158f1e5df2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:45.723376 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7f7f6f3e-239c-48bd-8b4d-52158f1e5df2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:45.724382 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7f7f6f3e-239c-48bd-8b4d-52158f1e5df2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:45.725265 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7f7f6f3e-239c-48bd-8b4d-52158f1e5df2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:45.931065 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-460be73f-e695-4288-8787-74ea15d1c613 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:45.932013 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-460be73f-e695-4288-8787-74ea15d1c613 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:45.939540 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7f7f6f3e-239c-48bd-8b4d-52158f1e5df2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:45.941926 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-460be73f-e695-4288-8787-74ea15d1c613 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:45.942875 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-460be73f-e695-4288-8787-74ea15d1c613 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:45.944374 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-460be73f-e695-4288-8787-74ea15d1c613 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:45.944975 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-460be73f-e695-4288-8787-74ea15d1c613 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:45.953041 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-7f7f6f3e-239c-48bd-8b4d-52158f1e5df2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1708 microversion: 2.1 time: 0.238091 Jul 11 14:42:45.955454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 171/362] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:45 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1708 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:46.020655 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-1c6a0c48-48d5-4670-98d4-a5bd9db45156 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:46.022007 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-1c6a0c48-48d5-4670-98d4-a5bd9db45156 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:46.032422 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1c6a0c48-48d5-4670-98d4-a5bd9db45156 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:46.033124 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1c6a0c48-48d5-4670-98d4-a5bd9db45156 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:46.034123 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1c6a0c48-48d5-4670-98d4-a5bd9db45156 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:46.035670 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1c6a0c48-48d5-4670-98d4-a5bd9db45156 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:46.318761 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-1c6a0c48-48d5-4670-98d4-a5bd9db45156 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:46.328958 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-1c6a0c48-48d5-4670-98d4-a5bd9db45156 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1555 microversion: 2.1 time: 0.317085 Jul 11 14:42:46.330244 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 172/363] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:46 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1555 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:46.351074 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-dcfbff48-ea01-4c4f-9cae-d0b7bdb41b32 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:46.352415 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-dcfbff48-ea01-4c4f-9cae-d0b7bdb41b32 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:46.360502 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dcfbff48-ea01-4c4f-9cae-d0b7bdb41b32 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:46.361242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dcfbff48-ea01-4c4f-9cae-d0b7bdb41b32 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:46.363545 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dcfbff48-ea01-4c4f-9cae-d0b7bdb41b32 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:46.364371 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dcfbff48-ea01-4c4f-9cae-d0b7bdb41b32 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:46.414898 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-460be73f-e695-4288-8787-74ea15d1c613 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:46.431164 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-460be73f-e695-4288-8787-74ea15d1c613 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:46.434321 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-460be73f-e695-4288-8787-74ea15d1c613 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:46.436828 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-460be73f-e695-4288-8787-74ea15d1c613 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1744 microversion: 2.20 time: 0.511431 Jul 11 14:42:46.440529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 192/364] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:45 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1744 bytes in 516 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:42:46.453657 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a2affc7c-af49-42a4-b928-770f8975c42c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:46.456320 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a2affc7c-af49-42a4-b928-770f8975c42c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:46.465123 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a2affc7c-af49-42a4-b928-770f8975c42c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:46.465562 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a2affc7c-af49-42a4-b928-770f8975c42c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:46.466342 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a2affc7c-af49-42a4-b928-770f8975c42c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:46.466981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a2affc7c-af49-42a4-b928-770f8975c42c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:46.535625 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-dcfbff48-ea01-4c4f-9cae-d0b7bdb41b32 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.193045 Jul 11 14:42:46.536840 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 173/365] 10.222.0.72 () {58 vars in 1274 bytes} [Mon Jul 11 14:42:46 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562/os-interface => generated 284 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:42:46.695212 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a2affc7c-af49-42a4-b928-770f8975c42c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:46.703153 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a2affc7c-af49-42a4-b928-770f8975c42c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1509 microversion: 2.1 time: 0.261022 Jul 11 14:42:46.703948 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 193/366] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:46 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1509 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:46.973872 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a6381913-4c63-4464-9593-8aaa41bb2e5c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:46.974815 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a6381913-4c63-4464-9593-8aaa41bb2e5c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:46.982365 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a6381913-4c63-4464-9593-8aaa41bb2e5c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:46.983157 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a6381913-4c63-4464-9593-8aaa41bb2e5c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:46.984116 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a6381913-4c63-4464-9593-8aaa41bb2e5c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:46.984723 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a6381913-4c63-4464-9593-8aaa41bb2e5c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:47.149942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-c97b1214-7723-4d55-b5e2-1beb8670a818 req-8279fc40-b042-4e56-8230-77135c6ab830 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3a91aed4-52ff-4220-bc8f-571c50452865", "tag": "957d5045-a8c1-4d28-8ee4-755331ae2c84"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:47.159757 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-c97b1214-7723-4d55-b5e2-1beb8670a818 req-8279fc40-b042-4e56-8230-77135c6ab830 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:47.160383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-c97b1214-7723-4d55-b5e2-1beb8670a818 req-8279fc40-b042-4e56-8230-77135c6ab830 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:47.176257 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-c97b1214-7723-4d55-b5e2-1beb8670a818 req-8279fc40-b042-4e56-8230-77135c6ab830 service nova] Creating event network-changed:957d5045-a8c1-4d28-8ee4-755331ae2c84 for instance 3a91aed4-52ff-4220-bc8f-571c50452865 on hv-cinder-maste Jul 11 14:42:47.181785 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-c97b1214-7723-4d55-b5e2-1beb8670a818 req-8279fc40-b042-4e56-8230-77135c6ab830 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036368 Jul 11 14:42:47.182618 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 194/367] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:42:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:42:47.395453 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a6381913-4c63-4464-9593-8aaa41bb2e5c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:47.404828 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a6381913-4c63-4464-9593-8aaa41bb2e5c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1708 microversion: 2.1 time: 0.434738 Jul 11 14:42:47.405662 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 174/368] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:46 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1708 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:47.460923 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f5423e76-c07f-4d16-9868-ff2df78ab76a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:47.461701 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f5423e76-c07f-4d16-9868-ff2df78ab76a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:47.467959 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f5423e76-c07f-4d16-9868-ff2df78ab76a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:47.468425 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f5423e76-c07f-4d16-9868-ff2df78ab76a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:47.469509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f5423e76-c07f-4d16-9868-ff2df78ab76a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:47.469642 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f5423e76-c07f-4d16-9868-ff2df78ab76a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:47.623174 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f5423e76-c07f-4d16-9868-ff2df78ab76a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:47.632016 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f5423e76-c07f-4d16-9868-ff2df78ab76a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:47.634210 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f5423e76-c07f-4d16-9868-ff2df78ab76a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:47.635460 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f5423e76-c07f-4d16-9868-ff2df78ab76a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1744 microversion: 2.20 time: 0.180233 Jul 11 14:42:47.636017 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 195/369] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:47 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1744 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:42:47.723648 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-dc6d3b3b-53a0-4d81-9265-d1d2bd8d9927 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:47.724322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-dc6d3b3b-53a0-4d81-9265-d1d2bd8d9927 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:47.733548 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dc6d3b3b-53a0-4d81-9265-d1d2bd8d9927 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:47.733998 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dc6d3b3b-53a0-4d81-9265-d1d2bd8d9927 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:47.734794 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dc6d3b3b-53a0-4d81-9265-d1d2bd8d9927 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:47.735406 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dc6d3b3b-53a0-4d81-9265-d1d2bd8d9927 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:47.946092 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-dc6d3b3b-53a0-4d81-9265-d1d2bd8d9927 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:47.953052 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-be7707a0-d01f-4c5e-9c68-424601b3f4dc req-913fe4e9-2d1e-49d8-a720-8f4173dd24dd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "13da005a-e085-43aa-8a18-1fe85e4e95af", "name": "network-vif-plugged", "status": "completed", "tag": "7b725b13-befd-4a14-98cc-9d0e599f5e17"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:47.966153 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-dc6d3b3b-53a0-4d81-9265-d1d2bd8d9927 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1509 microversion: 2.1 time: 0.248585 Jul 11 14:42:47.972018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 175/370] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:47 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1509 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:47.982309 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-be7707a0-d01f-4c5e-9c68-424601b3f4dc req-913fe4e9-2d1e-49d8-a720-8f4173dd24dd service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:47.983057 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-be7707a0-d01f-4c5e-9c68-424601b3f4dc req-913fe4e9-2d1e-49d8-a720-8f4173dd24dd service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:47.998023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-be7707a0-d01f-4c5e-9c68-424601b3f4dc req-913fe4e9-2d1e-49d8-a720-8f4173dd24dd service nova] Creating event network-vif-plugged:7b725b13-befd-4a14-98cc-9d0e599f5e17 for instance 13da005a-e085-43aa-8a18-1fe85e4e95af on hv-cinder-maste Jul 11 14:42:48.004920 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-be7707a0-d01f-4c5e-9c68-424601b3f4dc req-913fe4e9-2d1e-49d8-a720-8f4173dd24dd service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055763 Jul 11 14:42:48.005737 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 196/371] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:42:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:42:48.143346 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-50f63c4a-e332-476d-aee4-226c6dd83f02 req-8c2ca25c-977c-4020-99a2-8a22cb12cf43 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7a49d1ca-7491-41e0-b0d8-37c92b9d3562", "tag": "c62c0fe9-fd78-4a72-aef8-9bd42e6d81de"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:48.152521 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-50f63c4a-e332-476d-aee4-226c6dd83f02 req-8c2ca25c-977c-4020-99a2-8a22cb12cf43 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:48.152974 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-50f63c4a-e332-476d-aee4-226c6dd83f02 req-8c2ca25c-977c-4020-99a2-8a22cb12cf43 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:48.165227 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-50f63c4a-e332-476d-aee4-226c6dd83f02 req-8c2ca25c-977c-4020-99a2-8a22cb12cf43 service nova] Creating event network-changed:c62c0fe9-fd78-4a72-aef8-9bd42e6d81de for instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 on hv-cinder-maste Jul 11 14:42:48.171226 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-50f63c4a-e332-476d-aee4-226c6dd83f02 req-8c2ca25c-977c-4020-99a2-8a22cb12cf43 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031480 Jul 11 14:42:48.172017 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 176/372] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:42:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:42:48.422198 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-155bd800-c882-4a1a-8722-c3c46ccbd06c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:48.423051 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-155bd800-c882-4a1a-8722-c3c46ccbd06c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:48.429802 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-155bd800-c882-4a1a-8722-c3c46ccbd06c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:48.430325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-155bd800-c882-4a1a-8722-c3c46ccbd06c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:48.431106 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-155bd800-c882-4a1a-8722-c3c46ccbd06c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:48.431539 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-155bd800-c882-4a1a-8722-c3c46ccbd06c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:48.581174 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-155bd800-c882-4a1a-8722-c3c46ccbd06c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:48.590893 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-155bd800-c882-4a1a-8722-c3c46ccbd06c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1708 microversion: 2.1 time: 0.172810 Jul 11 14:42:48.600055 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 197/373] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:48 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1708 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:48.654391 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-6dc36bf7-b8bf-4ba6-b18d-173ecf08386c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:48.655127 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-6dc36bf7-b8bf-4ba6-b18d-173ecf08386c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:48.661695 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6dc36bf7-b8bf-4ba6-b18d-173ecf08386c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:48.662378 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6dc36bf7-b8bf-4ba6-b18d-173ecf08386c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:48.663184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6dc36bf7-b8bf-4ba6-b18d-173ecf08386c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:48.663646 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6dc36bf7-b8bf-4ba6-b18d-173ecf08386c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:48.800245 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6dc36bf7-b8bf-4ba6-b18d-173ecf08386c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:48.809658 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6dc36bf7-b8bf-4ba6-b18d-173ecf08386c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:48.811214 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6dc36bf7-b8bf-4ba6-b18d-173ecf08386c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:48.812405 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-6dc36bf7-b8bf-4ba6-b18d-173ecf08386c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1744 microversion: 2.20 time: 0.162489 Jul 11 14:42:48.812951 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 177/374] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:48 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1744 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:42:48.988087 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8c455a8b-1e56-4e78-97fb-f02c5ce48d79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:48.988797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8c455a8b-1e56-4e78-97fb-f02c5ce48d79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:48.995710 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8c455a8b-1e56-4e78-97fb-f02c5ce48d79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:48.996158 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8c455a8b-1e56-4e78-97fb-f02c5ce48d79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:48.997052 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8c455a8b-1e56-4e78-97fb-f02c5ce48d79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:48.997514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8c455a8b-1e56-4e78-97fb-f02c5ce48d79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:49.193200 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8c455a8b-1e56-4e78-97fb-f02c5ce48d79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:49.201274 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8c455a8b-1e56-4e78-97fb-f02c5ce48d79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1509 microversion: 2.1 time: 0.217447 Jul 11 14:42:49.201850 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 198/375] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:48 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1509 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:49.607710 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-94636c0e-48eb-4d9e-b16b-d29764e87a35 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:49.608361 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-94636c0e-48eb-4d9e-b16b-d29764e87a35 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:49.615249 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-94636c0e-48eb-4d9e-b16b-d29764e87a35 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:49.615704 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-94636c0e-48eb-4d9e-b16b-d29764e87a35 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:49.616481 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-94636c0e-48eb-4d9e-b16b-d29764e87a35 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:49.617421 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-94636c0e-48eb-4d9e-b16b-d29764e87a35 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:49.831687 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-89c4f518-4db6-4c05-8b00-d69004a91ae5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:49.832906 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-89c4f518-4db6-4c05-8b00-d69004a91ae5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:49.840715 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-89c4f518-4db6-4c05-8b00-d69004a91ae5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:49.841347 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-94636c0e-48eb-4d9e-b16b-d29764e87a35 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:49.841821 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-89c4f518-4db6-4c05-8b00-d69004a91ae5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:49.842203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-89c4f518-4db6-4c05-8b00-d69004a91ae5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:49.842926 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-89c4f518-4db6-4c05-8b00-d69004a91ae5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:49.854807 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-94636c0e-48eb-4d9e-b16b-d29764e87a35 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1708 microversion: 2.1 time: 0.250663 Jul 11 14:42:49.855491 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 178/376] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:49 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1708 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:49.985453 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-89c4f518-4db6-4c05-8b00-d69004a91ae5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:49.994972 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-89c4f518-4db6-4c05-8b00-d69004a91ae5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:49.996519 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-89c4f518-4db6-4c05-8b00-d69004a91ae5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:49.997664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-89c4f518-4db6-4c05-8b00-d69004a91ae5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1744 microversion: 2.20 time: 0.170216 Jul 11 14:42:49.998216 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 199/377] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:49 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1744 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:42:50.219374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3a100fee-1c80-4c6d-8889-6c3f418efd0b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:50.221049 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-3a100fee-1c80-4c6d-8889-6c3f418efd0b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:50.227278 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3a100fee-1c80-4c6d-8889-6c3f418efd0b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:50.227732 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3a100fee-1c80-4c6d-8889-6c3f418efd0b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:50.228628 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3a100fee-1c80-4c6d-8889-6c3f418efd0b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:50.229076 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3a100fee-1c80-4c6d-8889-6c3f418efd0b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:50.362021 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-3a100fee-1c80-4c6d-8889-6c3f418efd0b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:50.370317 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-3a100fee-1c80-4c6d-8889-6c3f418efd0b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1515 microversion: 2.1 time: 0.154904 Jul 11 14:42:50.370933 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 179/378] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:50 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1515 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:50.872913 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-b9dfac1d-0c61-45ea-b6eb-fc06651c661a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:50.873528 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-b9dfac1d-0c61-45ea-b6eb-fc06651c661a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:50.880180 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b9dfac1d-0c61-45ea-b6eb-fc06651c661a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:50.880630 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b9dfac1d-0c61-45ea-b6eb-fc06651c661a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:50.881387 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b9dfac1d-0c61-45ea-b6eb-fc06651c661a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:50.881816 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b9dfac1d-0c61-45ea-b6eb-fc06651c661a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:51.016398 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8a874261-8b77-40df-9715-e92931b7a3bf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:51.017011 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-8a874261-8b77-40df-9715-e92931b7a3bf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:51.024306 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-b9dfac1d-0c61-45ea-b6eb-fc06651c661a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:51.024493 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8a874261-8b77-40df-9715-e92931b7a3bf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:51.024945 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8a874261-8b77-40df-9715-e92931b7a3bf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:51.025895 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8a874261-8b77-40df-9715-e92931b7a3bf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:51.026338 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8a874261-8b77-40df-9715-e92931b7a3bf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:51.034134 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-b9dfac1d-0c61-45ea-b6eb-fc06651c661a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1712 microversion: 2.1 time: 0.164961 Jul 11 14:42:51.035400 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 200/379] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:50 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1712 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:51.174120 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8a874261-8b77-40df-9715-e92931b7a3bf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:51.182866 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8a874261-8b77-40df-9715-e92931b7a3bf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:51.184378 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8a874261-8b77-40df-9715-e92931b7a3bf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:51.185667 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8a874261-8b77-40df-9715-e92931b7a3bf tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1744 microversion: 2.20 time: 0.173153 Jul 11 14:42:51.186232 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 180/380] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:51 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1744 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:42:51.719041 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-6490aa71-fdea-4db9-9a86-227a2b3789e9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Action: 'create', calling method: .version_select at 0x7f59303099d0>, body: {"keypair": {"name": "tempest-keypair-126367978", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHOLdIdsYqkZWwPqwSd+7oy1CU4ExloH5OQxubUuBFhrMk5hiFCjJYg/DHCqr/RGcuUR4slhNdIMjv8d8l47TLC/xulIUvKNjvzPDB9s2Xrwb3i+LG18MOUOQLdMvFAOiA=="}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:51.768043 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-6490aa71-fdea-4db9-9a86-227a2b3789e9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Getting quotas for project 98966d44ea88477b94215761cdc92ea7. Resources: {'key_pairs'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:42:51.772734 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-6490aa71-fdea-4db9-9a86-227a2b3789e9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Getting quotas for user fa478fb4b41e49c5b1f9291f925cca23 and project 98966d44ea88477b94215761cdc92ea7. Resources: {'key_pairs'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:42:51.796383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-6490aa71-fdea-4db9-9a86-227a2b3789e9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.081163 Jul 11 14:42:51.797015 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 201/381] 10.222.0.72 () {60 vars in 1158 bytes} [Mon Jul 11 14:42:51 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:42:52.203843 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-cca3e5c8-4217-4929-b373-814f4b411e6e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:52.204455 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-cca3e5c8-4217-4929-b373-814f4b411e6e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:52.215408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cca3e5c8-4217-4929-b373-814f4b411e6e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:52.215874 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cca3e5c8-4217-4929-b373-814f4b411e6e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:52.216644 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cca3e5c8-4217-4929-b373-814f4b411e6e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:52.217079 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cca3e5c8-4217-4929-b373-814f4b411e6e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:52.448476 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-cca3e5c8-4217-4929-b373-814f4b411e6e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:52.457020 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-cca3e5c8-4217-4929-b373-814f4b411e6e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:52.458562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-cca3e5c8-4217-4929-b373-814f4b411e6e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:52.459728 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-cca3e5c8-4217-4929-b373-814f4b411e6e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1739 microversion: 2.20 time: 0.259734 Jul 11 14:42:52.460283 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 181/382] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:52 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1739 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:42:53.479696 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1f2550a6-73bf-4d8d-b440-819481c7d734 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:53.480811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1f2550a6-73bf-4d8d-b440-819481c7d734 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:53.514105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1f2550a6-73bf-4d8d-b440-819481c7d734 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:53.516639 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1f2550a6-73bf-4d8d-b440-819481c7d734 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:53.520856 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1f2550a6-73bf-4d8d-b440-819481c7d734 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:53.521713 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1f2550a6-73bf-4d8d-b440-819481c7d734 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:53.614340 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-416c872f-2d13-4a5f-8134-99b5e0318d2a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-8037567", "imageRef": "a588fa24-23b5-4e53-87c6-7e600ee3a5ef", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "382f35eb-d1cd-45e3-b28b-3e3389f544db"}], "security_groups": [{"name": "tempest-securitygroup--301783485"}], "key_name": "tempest-keypair-126367978", "user_data": "}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:42:53.742651 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1f2550a6-73bf-4d8d-b440-819481c7d734 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:53.751266 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1f2550a6-73bf-4d8d-b440-819481c7d734 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:53.757321 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1f2550a6-73bf-4d8d-b440-819481c7d734 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:53.759433 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1f2550a6-73bf-4d8d-b440-819481c7d734 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1739 microversion: 2.20 time: 0.284776 Jul 11 14:42:53.761254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 202/383] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:53 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1739 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:42:53.803392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.network.neutron [None req-416c872f-2d13-4a5f-8134-99b5e0318d2a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] validate_networks() for [('382f35eb-d1cd-45e3-b28b-3e3389f544db', None, None, None, None, None)] {{(pid=130160) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:42:54.368546 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-416c872f-2d13-4a5f-8134-99b5e0318d2a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Flavor limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:42:54.368990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-416c872f-2d13-4a5f-8134-99b5e0318d2a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Image limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:42:54.369478 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-416c872f-2d13-4a5f-8134-99b5e0318d2a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Flavor pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:42:54.369897 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-416c872f-2d13-4a5f-8134-99b5e0318d2a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Image pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:42:54.370319 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-416c872f-2d13-4a5f-8134-99b5e0318d2a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:42:54.379839 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-416c872f-2d13-4a5f-8134-99b5e0318d2a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:54.380571 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-416c872f-2d13-4a5f-8134-99b5e0318d2a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:54.381856 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-416c872f-2d13-4a5f-8134-99b5e0318d2a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:54.383061 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-416c872f-2d13-4a5f-8134-99b5e0318d2a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:54.405834 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-416c872f-2d13-4a5f-8134-99b5e0318d2a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Getting quotas for project 98966d44ea88477b94215761cdc92ea7. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:42:54.415256 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-416c872f-2d13-4a5f-8134-99b5e0318d2a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Getting quotas for user fa478fb4b41e49c5b1f9291f925cca23 and project 98966d44ea88477b94215761cdc92ea7. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:42:54.427720 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-416c872f-2d13-4a5f-8134-99b5e0318d2a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Going to run 1 instances... {{(pid=130160) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:42:54.430654 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-416c872f-2d13-4a5f-8134-99b5e0318d2a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a588fa24-23b5-4e53-87c6-7e600ee3a5ef',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130160) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:42:54.506085 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-416c872f-2d13-4a5f-8134-99b5e0318d2a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 0.895873 Jul 11 14:42:54.509986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 182/384] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:42:53 2022] POST /compute/v2.1/servers => generated 408 bytes in 900 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 11 14:42:54.521299 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-78d2c12d-8c82-4462-8a96-98d648c340dc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:54.522132 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-78d2c12d-8c82-4462-8a96-98d648c340dc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:54.609201 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-78d2c12d-8c82-4462-8a96-98d648c340dc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:54.633815 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-78d2c12d-8c82-4462-8a96-98d648c340dc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1242 microversion: 2.1 time: 0.117761 Jul 11 14:42:54.634418 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 203/385] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:54 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1242 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:54.781927 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-21c108e4-b57e-45a9-a95c-5579b29afa61 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:54.783334 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-21c108e4-b57e-45a9-a95c-5579b29afa61 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:54.805240 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-21c108e4-b57e-45a9-a95c-5579b29afa61 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:54.806014 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-21c108e4-b57e-45a9-a95c-5579b29afa61 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:54.807210 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-21c108e4-b57e-45a9-a95c-5579b29afa61 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:54.807810 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-21c108e4-b57e-45a9-a95c-5579b29afa61 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:55.367591 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-21c108e4-b57e-45a9-a95c-5579b29afa61 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:55.397927 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-21c108e4-b57e-45a9-a95c-5579b29afa61 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:55.403725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-21c108e4-b57e-45a9-a95c-5579b29afa61 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:55.406842 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-21c108e4-b57e-45a9-a95c-5579b29afa61 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1739 microversion: 2.20 time: 0.628400 Jul 11 14:42:55.408312 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 183/386] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:54 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1739 bytes in 632 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:42:55.651212 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-98d6aee8-371a-4fe1-9085-efb4cd3092e0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:55.652877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-98d6aee8-371a-4fe1-9085-efb4cd3092e0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:55.666737 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-98d6aee8-371a-4fe1-9085-efb4cd3092e0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:55.670875 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-98d6aee8-371a-4fe1-9085-efb4cd3092e0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:55.672190 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-98d6aee8-371a-4fe1-9085-efb4cd3092e0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:55.674122 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-98d6aee8-371a-4fe1-9085-efb4cd3092e0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:55.760559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-98d6aee8-371a-4fe1-9085-efb4cd3092e0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:55.768332 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-98d6aee8-371a-4fe1-9085-efb4cd3092e0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1246 microversion: 2.1 time: 0.122345 Jul 11 14:42:55.770874 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 204/387] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:55 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1246 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:56.436641 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7a892412-3531-4d9f-a167-7e1fc2c4b3a5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:56.438295 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-7a892412-3531-4d9f-a167-7e1fc2c4b3a5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:56.459149 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7a892412-3531-4d9f-a167-7e1fc2c4b3a5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:56.459614 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7a892412-3531-4d9f-a167-7e1fc2c4b3a5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:56.461552 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7a892412-3531-4d9f-a167-7e1fc2c4b3a5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:56.462887 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7a892412-3531-4d9f-a167-7e1fc2c4b3a5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:56.739212 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7a892412-3531-4d9f-a167-7e1fc2c4b3a5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:56.757785 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7a892412-3531-4d9f-a167-7e1fc2c4b3a5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:56.760924 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7a892412-3531-4d9f-a167-7e1fc2c4b3a5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:56.764430 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-7a892412-3531-4d9f-a167-7e1fc2c4b3a5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1739 microversion: 2.20 time: 0.336593 Jul 11 14:42:56.769753 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 184/388] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:56 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1739 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:42:56.790775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d859abe0-3cc5-45ff-826f-d72ca041e8d4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:56.792322 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d859abe0-3cc5-45ff-826f-d72ca041e8d4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:56.801020 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d859abe0-3cc5-45ff-826f-d72ca041e8d4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:56.801989 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d859abe0-3cc5-45ff-826f-d72ca041e8d4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:56.802872 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d859abe0-3cc5-45ff-826f-d72ca041e8d4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:56.803305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d859abe0-3cc5-45ff-826f-d72ca041e8d4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:56.905497 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d859abe0-3cc5-45ff-826f-d72ca041e8d4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:56.915260 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-d859abe0-3cc5-45ff-826f-d72ca041e8d4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1238 microversion: 2.1 time: 0.132472 Jul 11 14:42:56.915856 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 205/389] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:56 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1238 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:57.785813 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-55cdde34-21cd-4de3-96ff-b8cf4b220ad7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:57.787634 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-55cdde34-21cd-4de3-96ff-b8cf4b220ad7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:57.803214 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-55cdde34-21cd-4de3-96ff-b8cf4b220ad7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:57.805414 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-55cdde34-21cd-4de3-96ff-b8cf4b220ad7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:57.807621 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-55cdde34-21cd-4de3-96ff-b8cf4b220ad7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:57.808353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-55cdde34-21cd-4de3-96ff-b8cf4b220ad7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:57.935165 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-17f84873-4bf0-4385-815a-22a37e11c5ad tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:57.936631 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-17f84873-4bf0-4385-815a-22a37e11c5ad tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:57.950735 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-17f84873-4bf0-4385-815a-22a37e11c5ad tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:57.951335 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-17f84873-4bf0-4385-815a-22a37e11c5ad tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:57.952802 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-17f84873-4bf0-4385-815a-22a37e11c5ad tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:57.954902 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-17f84873-4bf0-4385-815a-22a37e11c5ad tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:58.150981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-17f84873-4bf0-4385-815a-22a37e11c5ad tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:58.167543 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-17f84873-4bf0-4385-815a-22a37e11c5ad tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1302 microversion: 2.1 time: 0.234410 Jul 11 14:42:58.180888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 206/390] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:57 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1302 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:42:58.392549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-55cdde34-21cd-4de3-96ff-b8cf4b220ad7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:58.405541 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-55cdde34-21cd-4de3-96ff-b8cf4b220ad7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:58.410000 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-55cdde34-21cd-4de3-96ff-b8cf4b220ad7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:58.419065 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-55cdde34-21cd-4de3-96ff-b8cf4b220ad7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1739 microversion: 2.20 time: 0.635550 Jul 11 14:42:58.434033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 185/391] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:57 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1739 bytes in 652 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:42:59.187590 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-0d29f4a3-fcf6-4415-ac1f-4877b7f4de39 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:59.188794 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-0d29f4a3-fcf6-4415-ac1f-4877b7f4de39 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:59.206378 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0d29f4a3-fcf6-4415-ac1f-4877b7f4de39 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:59.207215 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0d29f4a3-fcf6-4415-ac1f-4877b7f4de39 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:59.208236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0d29f4a3-fcf6-4415-ac1f-4877b7f4de39 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:59.208857 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0d29f4a3-fcf6-4415-ac1f-4877b7f4de39 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:59.323331 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0d29f4a3-fcf6-4415-ac1f-4877b7f4de39 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:59.333386 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-0d29f4a3-fcf6-4415-ac1f-4877b7f4de39 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1300 microversion: 2.1 time: 0.150731 Jul 11 14:42:59.334772 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 207/392] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:42:59 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1300 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:42:59.433052 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-90751ce3-13dd-4ed2-85a2-66702acd7c9a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:42:59.433601 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-90751ce3-13dd-4ed2-85a2-66702acd7c9a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:42:59.447598 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-90751ce3-13dd-4ed2-85a2-66702acd7c9a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:59.448022 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-90751ce3-13dd-4ed2-85a2-66702acd7c9a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:59.448797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-90751ce3-13dd-4ed2-85a2-66702acd7c9a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:42:59.449241 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-90751ce3-13dd-4ed2-85a2-66702acd7c9a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:42:59.801828 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-90751ce3-13dd-4ed2-85a2-66702acd7c9a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:59.817850 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-90751ce3-13dd-4ed2-85a2-66702acd7c9a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:59.819430 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-90751ce3-13dd-4ed2-85a2-66702acd7c9a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:42:59.829864 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-90751ce3-13dd-4ed2-85a2-66702acd7c9a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1739 microversion: 2.20 time: 0.392107 Jul 11 14:42:59.847686 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 186/393] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:42:59 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1739 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:00.350262 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e78795c3-5a56-49ba-a761-4d651172cf3e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:00.352110 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e78795c3-5a56-49ba-a761-4d651172cf3e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:00.358692 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e78795c3-5a56-49ba-a761-4d651172cf3e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:00.375197 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e78795c3-5a56-49ba-a761-4d651172cf3e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:00.376337 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e78795c3-5a56-49ba-a761-4d651172cf3e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:00.376948 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e78795c3-5a56-49ba-a761-4d651172cf3e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:00.852492 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-734e0e15-c39d-46bd-8807-41dee5539013 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:00.855768 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-734e0e15-c39d-46bd-8807-41dee5539013 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:00.868847 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-734e0e15-c39d-46bd-8807-41dee5539013 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:00.869710 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-734e0e15-c39d-46bd-8807-41dee5539013 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:00.870765 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-734e0e15-c39d-46bd-8807-41dee5539013 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:00.871444 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-734e0e15-c39d-46bd-8807-41dee5539013 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:01.008090 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e78795c3-5a56-49ba-a761-4d651172cf3e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:01.016969 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e78795c3-5a56-49ba-a761-4d651172cf3e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1367 microversion: 2.1 time: 0.670539 Jul 11 14:43:01.025582 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 208/394] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:00 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1367 bytes in 680 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:01.096863 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-734e0e15-c39d-46bd-8807-41dee5539013 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:01.100512 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=100 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:01.109292 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-734e0e15-c39d-46bd-8807-41dee5539013 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:01.111166 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-734e0e15-c39d-46bd-8807-41dee5539013 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:01.112639 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-734e0e15-c39d-46bd-8807-41dee5539013 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1739 microversion: 2.20 time: 0.264793 Jul 11 14:43:01.113462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 187/395] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:00 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1739 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:43:02.042659 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-22f63c14-4458-44d6-8440-db534e1c7cca tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:02.044051 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-22f63c14-4458-44d6-8440-db534e1c7cca tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:02.056146 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-22f63c14-4458-44d6-8440-db534e1c7cca tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:02.057416 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-22f63c14-4458-44d6-8440-db534e1c7cca tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:02.064620 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-22f63c14-4458-44d6-8440-db534e1c7cca tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:02.065842 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-22f63c14-4458-44d6-8440-db534e1c7cca tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:02.148497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7a13c39e-f7ae-4dba-9cf5-5af452344a3f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:02.150010 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-7a13c39e-f7ae-4dba-9cf5-5af452344a3f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:02.157770 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7a13c39e-f7ae-4dba-9cf5-5af452344a3f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:02.158783 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7a13c39e-f7ae-4dba-9cf5-5af452344a3f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:02.159793 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7a13c39e-f7ae-4dba-9cf5-5af452344a3f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:02.160436 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7a13c39e-f7ae-4dba-9cf5-5af452344a3f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:02.308752 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-22f63c14-4458-44d6-8440-db534e1c7cca tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:02.332063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-22f63c14-4458-44d6-8440-db534e1c7cca tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1367 microversion: 2.1 time: 0.293760 Jul 11 14:43:02.333351 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 209/396] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:02 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1367 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:02.985802 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7a13c39e-f7ae-4dba-9cf5-5af452344a3f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:03.014885 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7a13c39e-f7ae-4dba-9cf5-5af452344a3f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:03.019430 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7a13c39e-f7ae-4dba-9cf5-5af452344a3f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:03.023993 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-7a13c39e-f7ae-4dba-9cf5-5af452344a3f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1739 microversion: 2.20 time: 0.889809 Jul 11 14:43:03.026200 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 188/397] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:02 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1739 bytes in 893 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:43:03.263037 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-8172f192-0f5b-406e-bc71-5a8583e1e133 req-f8bd564f-fcbf-4ec0-ba02-a74ab951dd11 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f66823cd-4fe8-42cf-ab7a-d7cd7afc289a", "tag": "5a30e035-3f54-418c-8ac6-380a40fc2f5d"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:43:03.275161 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=99,nova_api:INSERT=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:03.294355 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-8172f192-0f5b-406e-bc71-5a8583e1e133 req-f8bd564f-fcbf-4ec0-ba02-a74ab951dd11 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:03.298920 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-8172f192-0f5b-406e-bc71-5a8583e1e133 req-f8bd564f-fcbf-4ec0-ba02-a74ab951dd11 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:03.332560 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-8172f192-0f5b-406e-bc71-5a8583e1e133 req-f8bd564f-fcbf-4ec0-ba02-a74ab951dd11 service nova] Creating event network-changed:5a30e035-3f54-418c-8ac6-380a40fc2f5d for instance f66823cd-4fe8-42cf-ab7a-d7cd7afc289a on hv-cinder-maste Jul 11 14:43:03.349071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7fda390c-faf3-4545-8867-69d5a9f23fb3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:03.349684 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-7fda390c-faf3-4545-8867-69d5a9f23fb3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:03.358690 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7fda390c-faf3-4545-8867-69d5a9f23fb3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:03.359137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7fda390c-faf3-4545-8867-69d5a9f23fb3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:03.359755 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-8172f192-0f5b-406e-bc71-5a8583e1e133 req-f8bd564f-fcbf-4ec0-ba02-a74ab951dd11 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.102256 Jul 11 14:43:03.360850 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 210/398] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:43:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:43:03.366318 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7fda390c-faf3-4545-8867-69d5a9f23fb3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:03.368360 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7fda390c-faf3-4545-8867-69d5a9f23fb3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:03.691993 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7fda390c-faf3-4545-8867-69d5a9f23fb3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:03.703571 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-7fda390c-faf3-4545-8867-69d5a9f23fb3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1367 microversion: 2.1 time: 0.357313 Jul 11 14:43:03.714874 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 189/399] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:03 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1367 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:04.047008 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3ae37ad6-6bf8-43df-aa47-801d569009b8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:04.053083 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3ae37ad6-6bf8-43df-aa47-801d569009b8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:04.070410 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3ae37ad6-6bf8-43df-aa47-801d569009b8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:04.077536 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3ae37ad6-6bf8-43df-aa47-801d569009b8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:04.078833 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3ae37ad6-6bf8-43df-aa47-801d569009b8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:04.080006 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3ae37ad6-6bf8-43df-aa47-801d569009b8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:04.429803 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3ae37ad6-6bf8-43df-aa47-801d569009b8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:04.441506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3ae37ad6-6bf8-43df-aa47-801d569009b8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:04.448772 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3ae37ad6-6bf8-43df-aa47-801d569009b8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:04.451559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3ae37ad6-6bf8-43df-aa47-801d569009b8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1739 microversion: 2.20 time: 0.408913 Jul 11 14:43:04.455478 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 211/400] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:04 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1739 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:04.723267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-5124950a-18a8-4c3c-b927-3ba8896d2ef5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:04.724563 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-5124950a-18a8-4c3c-b927-3ba8896d2ef5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:04.735248 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=2 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:04.736874 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=6 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:04.740225 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5124950a-18a8-4c3c-b927-3ba8896d2ef5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:04.741087 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5124950a-18a8-4c3c-b927-3ba8896d2ef5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:04.742139 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5124950a-18a8-4c3c-b927-3ba8896d2ef5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:04.742827 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5124950a-18a8-4c3c-b927-3ba8896d2ef5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:05.043509 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5124950a-18a8-4c3c-b927-3ba8896d2ef5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:05.056518 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-5124950a-18a8-4c3c-b927-3ba8896d2ef5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1367 microversion: 2.1 time: 0.337014 Jul 11 14:43:05.057357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 190/401] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:04 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1367 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:05.477532 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-0946680a-2ee1-468e-a361-e85e0a42ad68 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:05.480289 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-0946680a-2ee1-468e-a361-e85e0a42ad68 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:05.482816 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=6 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:05.497692 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0946680a-2ee1-468e-a361-e85e0a42ad68 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:05.498599 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0946680a-2ee1-468e-a361-e85e0a42ad68 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:05.499704 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0946680a-2ee1-468e-a361-e85e0a42ad68 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:05.500373 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0946680a-2ee1-468e-a361-e85e0a42ad68 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:05.513892 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:05.974718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0946680a-2ee1-468e-a361-e85e0a42ad68 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:05.988622 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0946680a-2ee1-468e-a361-e85e0a42ad68 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:05.996507 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0946680a-2ee1-468e-a361-e85e0a42ad68 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:05.998223 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-0946680a-2ee1-468e-a361-e85e0a42ad68 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1739 microversion: 2.20 time: 0.528935 Jul 11 14:43:05.998893 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 212/402] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:05 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1739 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:06.075206 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-669aa715-1f12-46db-9198-1a4084dc1488 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:06.075997 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-669aa715-1f12-46db-9198-1a4084dc1488 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:06.087811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-669aa715-1f12-46db-9198-1a4084dc1488 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:06.092977 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-669aa715-1f12-46db-9198-1a4084dc1488 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:06.094220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-669aa715-1f12-46db-9198-1a4084dc1488 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:06.107720 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-669aa715-1f12-46db-9198-1a4084dc1488 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:06.537833 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-669aa715-1f12-46db-9198-1a4084dc1488 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:06.551275 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-669aa715-1f12-46db-9198-1a4084dc1488 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1367 microversion: 2.1 time: 0.479176 Jul 11 14:43:06.552775 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 191/403] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:06 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1367 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:07.042760 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-606604f3-dc1a-45c2-9645-8f7aca5cf100 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:07.043445 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-606604f3-dc1a-45c2-9645-8f7aca5cf100 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:07.075449 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-606604f3-dc1a-45c2-9645-8f7aca5cf100 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:07.075899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-606604f3-dc1a-45c2-9645-8f7aca5cf100 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:07.076682 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-606604f3-dc1a-45c2-9645-8f7aca5cf100 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:07.077125 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-606604f3-dc1a-45c2-9645-8f7aca5cf100 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:07.367520 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-606604f3-dc1a-45c2-9645-8f7aca5cf100 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:07.384325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-606604f3-dc1a-45c2-9645-8f7aca5cf100 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:07.389833 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-606604f3-dc1a-45c2-9645-8f7aca5cf100 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:07.399479 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-606604f3-dc1a-45c2-9645-8f7aca5cf100 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1739 microversion: 2.20 time: 0.385390 Jul 11 14:43:07.400452 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 213/404] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:07 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1739 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:07.568594 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-66af1cc2-e564-4335-ab63-65a8ffdfa05f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:07.570412 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-66af1cc2-e564-4335-ab63-65a8ffdfa05f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:07.583075 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-66af1cc2-e564-4335-ab63-65a8ffdfa05f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:07.583529 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-66af1cc2-e564-4335-ab63-65a8ffdfa05f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:07.584373 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-66af1cc2-e564-4335-ab63-65a8ffdfa05f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:07.584740 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-66af1cc2-e564-4335-ab63-65a8ffdfa05f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:07.888764 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-66af1cc2-e564-4335-ab63-65a8ffdfa05f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:07.908914 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-66af1cc2-e564-4335-ab63-65a8ffdfa05f tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1367 microversion: 2.1 time: 0.344127 Jul 11 14:43:07.912012 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 192/405] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:07 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1367 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:08.423475 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-68b90fdd-e181-486d-87c3-7fa051567325 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:08.424801 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-68b90fdd-e181-486d-87c3-7fa051567325 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:08.441910 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-68b90fdd-e181-486d-87c3-7fa051567325 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:08.445482 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-68b90fdd-e181-486d-87c3-7fa051567325 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:08.446717 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-68b90fdd-e181-486d-87c3-7fa051567325 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:08.447611 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-68b90fdd-e181-486d-87c3-7fa051567325 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:08.674558 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-68b90fdd-e181-486d-87c3-7fa051567325 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:08.687364 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-68b90fdd-e181-486d-87c3-7fa051567325 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:08.688936 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-68b90fdd-e181-486d-87c3-7fa051567325 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:08.690811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-68b90fdd-e181-486d-87c3-7fa051567325 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1739 microversion: 2.20 time: 0.273693 Jul 11 14:43:08.691879 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 214/406] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:08 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1739 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:08.930522 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d98c9a62-6513-463f-aa21-db31b49dc968 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:08.931243 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d98c9a62-6513-463f-aa21-db31b49dc968 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:08.940084 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d98c9a62-6513-463f-aa21-db31b49dc968 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:08.940650 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d98c9a62-6513-463f-aa21-db31b49dc968 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:08.941478 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d98c9a62-6513-463f-aa21-db31b49dc968 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:08.942046 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d98c9a62-6513-463f-aa21-db31b49dc968 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:09.301661 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d98c9a62-6513-463f-aa21-db31b49dc968 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:09.310641 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d98c9a62-6513-463f-aa21-db31b49dc968 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1367 microversion: 2.1 time: 0.385673 Jul 11 14:43:09.320167 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 193/407] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:08 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1367 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:09.710048 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-b7bb9108-7247-450b-abdb-c4368f4bbfdb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:09.711320 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-b7bb9108-7247-450b-abdb-c4368f4bbfdb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:09.724896 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b7bb9108-7247-450b-abdb-c4368f4bbfdb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:09.726178 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b7bb9108-7247-450b-abdb-c4368f4bbfdb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:09.729462 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b7bb9108-7247-450b-abdb-c4368f4bbfdb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:09.729918 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b7bb9108-7247-450b-abdb-c4368f4bbfdb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:10.047260 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-b7bb9108-7247-450b-abdb-c4368f4bbfdb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:10.058955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-b7bb9108-7247-450b-abdb-c4368f4bbfdb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:10.061040 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-b7bb9108-7247-450b-abdb-c4368f4bbfdb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:10.062882 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-b7bb9108-7247-450b-abdb-c4368f4bbfdb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1721 microversion: 2.20 time: 0.357140 Jul 11 14:43:10.063983 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 215/408] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:09 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1721 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:10.327555 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-56874eac-1c17-4bd0-963a-8ec8b565c3d4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:10.329066 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-56874eac-1c17-4bd0-963a-8ec8b565c3d4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:10.333439 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=3 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:10.339216 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-56874eac-1c17-4bd0-963a-8ec8b565c3d4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:10.339885 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-56874eac-1c17-4bd0-963a-8ec8b565c3d4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:10.340676 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-56874eac-1c17-4bd0-963a-8ec8b565c3d4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:10.342413 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-56874eac-1c17-4bd0-963a-8ec8b565c3d4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:10.588618 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-56874eac-1c17-4bd0-963a-8ec8b565c3d4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:10.606302 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-56874eac-1c17-4bd0-963a-8ec8b565c3d4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1367 microversion: 2.1 time: 0.282151 Jul 11 14:43:10.607476 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 194/409] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:10 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1367 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:10.829878 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-15dde71c-b4df-40c6-94e8-10cf50ff63dd tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:10.830559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-15dde71c-b4df-40c6-94e8-10cf50ff63dd tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:10.838687 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-15dde71c-b4df-40c6-94e8-10cf50ff63dd tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:10.839395 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-15dde71c-b4df-40c6-94e8-10cf50ff63dd tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:10.840420 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-15dde71c-b4df-40c6-94e8-10cf50ff63dd tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:10.841080 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-15dde71c-b4df-40c6-94e8-10cf50ff63dd tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:11.035298 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.objects.instance [None req-15dde71c-b4df-40c6-94e8-10cf50ff63dd tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lazy-loading 'services' on Instance uuid 751f3bb7-cb4e-43c3-aab8-554e34dc828c {{(pid=130159) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:43:11.081842 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c0d3bd93-fe31-4fe9-a6d6-5ae73fd9f910 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:11.082795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c0d3bd93-fe31-4fe9-a6d6-5ae73fd9f910 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:11.091378 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c0d3bd93-fe31-4fe9-a6d6-5ae73fd9f910 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:11.091836 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c0d3bd93-fe31-4fe9-a6d6-5ae73fd9f910 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:11.092609 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c0d3bd93-fe31-4fe9-a6d6-5ae73fd9f910 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:11.098057 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c0d3bd93-fe31-4fe9-a6d6-5ae73fd9f910 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:11.198580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-15dde71c-b4df-40c6-94e8-10cf50ff63dd tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "DELETE /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c/os-volume_attachments/fee0dc64-49de-4fe5-8b94-881d049f6d4b" status: 202 len: 0 microversion: 2.1 time: 0.372168 Jul 11 14:43:11.199819 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 216/410] 10.222.0.72 () {58 vars in 1415 bytes} [Mon Jul 11 14:43:10 2022] DELETE /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c/os-volume_attachments/fee0dc64-49de-4fe5-8b94-881d049f6d4b => generated 0 bytes in 374 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:43:11.310831 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c0d3bd93-fe31-4fe9-a6d6-5ae73fd9f910 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:11.321893 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c0d3bd93-fe31-4fe9-a6d6-5ae73fd9f910 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:11.323895 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c0d3bd93-fe31-4fe9-a6d6-5ae73fd9f910 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:11.325319 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c0d3bd93-fe31-4fe9-a6d6-5ae73fd9f910 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1721 microversion: 2.20 time: 0.247802 Jul 11 14:43:11.338300 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 195/411] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:11 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1721 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:43:11.624003 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f359d2c8-c74c-4ae7-9821-690b50487f5b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:11.625065 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f359d2c8-c74c-4ae7-9821-690b50487f5b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:11.633800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f359d2c8-c74c-4ae7-9821-690b50487f5b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:11.634684 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f359d2c8-c74c-4ae7-9821-690b50487f5b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:11.635537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f359d2c8-c74c-4ae7-9821-690b50487f5b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:11.636074 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f359d2c8-c74c-4ae7-9821-690b50487f5b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:11.802791 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f359d2c8-c74c-4ae7-9821-690b50487f5b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:11.811043 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f359d2c8-c74c-4ae7-9821-690b50487f5b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1367 microversion: 2.1 time: 0.191203 Jul 11 14:43:11.820951 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 217/412] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:11 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1367 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:12.345044 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a26bb433-ea60-42a8-b75b-5d908c314e90 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:12.345655 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a26bb433-ea60-42a8-b75b-5d908c314e90 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:12.353421 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a26bb433-ea60-42a8-b75b-5d908c314e90 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:12.355982 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a26bb433-ea60-42a8-b75b-5d908c314e90 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:12.359531 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a26bb433-ea60-42a8-b75b-5d908c314e90 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:12.360485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a26bb433-ea60-42a8-b75b-5d908c314e90 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:12.728197 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a26bb433-ea60-42a8-b75b-5d908c314e90 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:12.729752 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-e05ba972-4496-431b-b60d-580f42427d32 req-6a83a3c0-47db-4d4b-9861-89ed21626819 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f66823cd-4fe8-42cf-ab7a-d7cd7afc289a", "name": "network-vif-plugged", "status": "completed", "tag": "5a30e035-3f54-418c-8ac6-380a40fc2f5d"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:43:12.744432 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a26bb433-ea60-42a8-b75b-5d908c314e90 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:12.748263 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-e05ba972-4496-431b-b60d-580f42427d32 req-6a83a3c0-47db-4d4b-9861-89ed21626819 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:12.748728 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-e05ba972-4496-431b-b60d-580f42427d32 req-6a83a3c0-47db-4d4b-9861-89ed21626819 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:12.752170 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a26bb433-ea60-42a8-b75b-5d908c314e90 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:12.755152 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a26bb433-ea60-42a8-b75b-5d908c314e90 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1721 microversion: 2.20 time: 0.414004 Jul 11 14:43:12.767462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 196/413] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:12 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1721 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:12.785053 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-e05ba972-4496-431b-b60d-580f42427d32 req-6a83a3c0-47db-4d4b-9861-89ed21626819 service nova] Creating event network-vif-plugged:5a30e035-3f54-418c-8ac6-380a40fc2f5d for instance f66823cd-4fe8-42cf-ab7a-d7cd7afc289a on hv-cinder-maste Jul 11 14:43:12.796555 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-e05ba972-4496-431b-b60d-580f42427d32 req-6a83a3c0-47db-4d4b-9861-89ed21626819 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071642 Jul 11 14:43:12.797427 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 218/414] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:43:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (3 switches on core 0) Jul 11 14:43:12.833565 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8d37f33c-7656-475e-9918-4f2761b1f5fa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:12.834810 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-8d37f33c-7656-475e-9918-4f2761b1f5fa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:12.844682 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d37f33c-7656-475e-9918-4f2761b1f5fa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:12.845541 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d37f33c-7656-475e-9918-4f2761b1f5fa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:12.846669 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d37f33c-7656-475e-9918-4f2761b1f5fa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:12.847705 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d37f33c-7656-475e-9918-4f2761b1f5fa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:13.156757 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8d37f33c-7656-475e-9918-4f2761b1f5fa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:13.171757 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8d37f33c-7656-475e-9918-4f2761b1f5fa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1367 microversion: 2.1 time: 0.345405 Jul 11 14:43:13.180880 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 197/415] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:12 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1367 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:13.785866 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9ed4ee52-42b1-4628-94e3-93ae8cddb78f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:13.786681 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9ed4ee52-42b1-4628-94e3-93ae8cddb78f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:13.804423 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9ed4ee52-42b1-4628-94e3-93ae8cddb78f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:13.804882 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9ed4ee52-42b1-4628-94e3-93ae8cddb78f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:13.805806 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9ed4ee52-42b1-4628-94e3-93ae8cddb78f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:13.806236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9ed4ee52-42b1-4628-94e3-93ae8cddb78f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:13.994966 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9ed4ee52-42b1-4628-94e3-93ae8cddb78f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:14.005443 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9ed4ee52-42b1-4628-94e3-93ae8cddb78f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:14.007847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9ed4ee52-42b1-4628-94e3-93ae8cddb78f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:14.009052 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9ed4ee52-42b1-4628-94e3-93ae8cddb78f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1724 microversion: 2.20 time: 0.227544 Jul 11 14:43:14.010194 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 219/416] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:13 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1724 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:14.027956 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-52fe5a9c-8ce1-4b2f-83a0-011b4b9710e5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ced20d51-419d-4832-a700-5c8fa50339ab"}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:43:14.029390 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-52fe5a9c-8ce1-4b2f-83a0-011b4b9710e5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:14.039242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-52fe5a9c-8ce1-4b2f-83a0-011b4b9710e5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:14.040000 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-52fe5a9c-8ce1-4b2f-83a0-011b4b9710e5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:14.041020 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-52fe5a9c-8ce1-4b2f-83a0-011b4b9710e5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:14.041686 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-52fe5a9c-8ce1-4b2f-83a0-011b4b9710e5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:14.190678 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-65843df5-28cd-4aa6-83f8-0f7e01fa16e8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:14.191390 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-65843df5-28cd-4aa6-83f8-0f7e01fa16e8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:14.204782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-65843df5-28cd-4aa6-83f8-0f7e01fa16e8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:14.205422 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-65843df5-28cd-4aa6-83f8-0f7e01fa16e8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:14.215452 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-65843df5-28cd-4aa6-83f8-0f7e01fa16e8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:14.216218 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-65843df5-28cd-4aa6-83f8-0f7e01fa16e8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:14.432032 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-65843df5-28cd-4aa6-83f8-0f7e01fa16e8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:14.440340 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-65843df5-28cd-4aa6-83f8-0f7e01fa16e8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1367 microversion: 2.1 time: 0.255175 Jul 11 14:43:14.442421 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 220/417] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:14 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1367 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:14.513036 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-52fe5a9c-8ce1-4b2f-83a0-011b4b9710e5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "POST /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.489026 Jul 11 14:43:14.514579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 198/418] 10.222.0.72 () {62 vars in 1363 bytes} [Mon Jul 11 14:43:14 2022] POST /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/os-volume_attachments => generated 188 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:43:14.609253 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-076892f7-5022-4c58-89ba-89ce950d366f tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:14.609858 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-076892f7-5022-4c58-89ba-89ce950d366f tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:14.624043 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-076892f7-5022-4c58-89ba-89ce950d366f tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:14.625527 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-076892f7-5022-4c58-89ba-89ce950d366f tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:14.627807 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-076892f7-5022-4c58-89ba-89ce950d366f tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:14.628995 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-076892f7-5022-4c58-89ba-89ce950d366f tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:14.712561 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-52839f7f-74f9-411e-9c95-4c7b412fd48e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:43:14.713342 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-52839f7f-74f9-411e-9c95-4c7b412fd48e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:14.714593 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-076892f7-5022-4c58-89ba-89ce950d366f tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Going to try to terminate instance {{(pid=130159) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:43:14.727871 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-52839f7f-74f9-411e-9c95-4c7b412fd48e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:14.732168 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-52839f7f-74f9-411e-9c95-4c7b412fd48e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:14.733195 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-52839f7f-74f9-411e-9c95-4c7b412fd48e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:14.734797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-52839f7f-74f9-411e-9c95-4c7b412fd48e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:14.893449 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-076892f7-5022-4c58-89ba-89ce950d366f tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "DELETE /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c" status: 204 len: 0 microversion: 2.1 time: 0.293788 Jul 11 14:43:14.895853 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 221/419] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:43:14 2022] DELETE /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 11 14:43:14.921763 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-14a0cdf2-7f22-4c83-8355-896ebe525368 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:14.922658 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-14a0cdf2-7f22-4c83-8355-896ebe525368 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:14.945295 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-14a0cdf2-7f22-4c83-8355-896ebe525368 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:14.946068 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-14a0cdf2-7f22-4c83-8355-896ebe525368 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:14.947251 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-14a0cdf2-7f22-4c83-8355-896ebe525368 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:14.950781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-14a0cdf2-7f22-4c83-8355-896ebe525368 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:15.001337 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-52839f7f-74f9-411e-9c95-4c7b412fd48e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "POST /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/action" status: 202 len: 0 microversion: 2.20 time: 0.291431 Jul 11 14:43:15.002597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 199/420] 10.222.0.72 () {62 vars in 1318 bytes} [Mon Jul 11 14:43:14 2022] POST /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/action => generated 0 bytes in 295 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Jul 11 14:43:15.015315 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-2d39436d-8edd-45e1-9ced-81bfee0393e3 req-3f9d09dd-8a86-4f16-b787-516c5c4cd076 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "13da005a-e085-43aa-8a18-1fe85e4e95af", "name": "network-vif-unplugged", "status": "completed", "tag": "7b725b13-befd-4a14-98cc-9d0e599f5e17"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:43:15.031238 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-2d39436d-8edd-45e1-9ced-81bfee0393e3 req-3f9d09dd-8a86-4f16-b787-516c5c4cd076 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:15.032572 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-2d39436d-8edd-45e1-9ced-81bfee0393e3 req-3f9d09dd-8a86-4f16-b787-516c5c4cd076 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:15.050502 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.compute.server_external_events [req-2d39436d-8edd-45e1-9ced-81bfee0393e3 req-3f9d09dd-8a86-4f16-b787-516c5c4cd076 service nova] Unable to find a host for instance 13da005a-e085-43aa-8a18-1fe85e4e95af. Dropping event network-vif-unplugged {{(pid=130160) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:143}} Jul 11 14:43:15.051820 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-2d39436d-8edd-45e1-9ced-81bfee0393e3 req-3f9d09dd-8a86-4f16-b787-516c5c4cd076 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.048249 Jul 11 14:43:15.052782 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 200/421] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:43:15 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 50 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Jul 11 14:43:15.062349 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c0994cad-4a17-47dd-b765-058b81e0ec95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:15.063314 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c0994cad-4a17-47dd-b765-058b81e0ec95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:15.071313 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c0994cad-4a17-47dd-b765-058b81e0ec95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:15.072038 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c0994cad-4a17-47dd-b765-058b81e0ec95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:15.073005 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c0994cad-4a17-47dd-b765-058b81e0ec95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:15.073591 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c0994cad-4a17-47dd-b765-058b81e0ec95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:15.102270 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:15.302681 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c0994cad-4a17-47dd-b765-058b81e0ec95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:15.315969 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c0994cad-4a17-47dd-b765-058b81e0ec95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:15.319550 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c0994cad-4a17-47dd-b765-058b81e0ec95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:15.321308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c0994cad-4a17-47dd-b765-058b81e0ec95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1750 microversion: 2.20 time: 0.266982 Jul 11 14:43:15.322374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 201/422] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:15 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1750 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:15.351303 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-14a0cdf2-7f22-4c83-8355-896ebe525368 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:15.362979 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-14a0cdf2-7f22-4c83-8355-896ebe525368 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c" status: 200 len: 1514 microversion: 2.1 time: 0.461294 Jul 11 14:43:15.363689 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 222/423] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:14 2022] GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c => generated 1514 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:15.461277 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-ec11115f-a730-4954-a3ce-9f2317aaef25 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:15.462400 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-ec11115f-a730-4954-a3ce-9f2317aaef25 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:15.472041 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ec11115f-a730-4954-a3ce-9f2317aaef25 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:15.473054 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ec11115f-a730-4954-a3ce-9f2317aaef25 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:15.473845 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ec11115f-a730-4954-a3ce-9f2317aaef25 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:15.474465 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ec11115f-a730-4954-a3ce-9f2317aaef25 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:15.713256 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-ec11115f-a730-4954-a3ce-9f2317aaef25 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:15.722098 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-ec11115f-a730-4954-a3ce-9f2317aaef25 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1367 microversion: 2.1 time: 0.266893 Jul 11 14:43:15.723918 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 202/424] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:15 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1367 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:16.340388 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9b55101b-ae19-4f69-89b8-d6db27c75cb3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:16.342808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9b55101b-ae19-4f69-89b8-d6db27c75cb3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:16.354545 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9b55101b-ae19-4f69-89b8-d6db27c75cb3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:16.355231 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9b55101b-ae19-4f69-89b8-d6db27c75cb3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:16.356318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9b55101b-ae19-4f69-89b8-d6db27c75cb3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:16.357030 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9b55101b-ae19-4f69-89b8-d6db27c75cb3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:16.380731 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-cb854182-cf2e-428f-ac1f-df39a4a9125a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:16.381345 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-cb854182-cf2e-428f-ac1f-df39a4a9125a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:16.395976 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cb854182-cf2e-428f-ac1f-df39a4a9125a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:16.396420 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cb854182-cf2e-428f-ac1f-df39a4a9125a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:16.400774 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cb854182-cf2e-428f-ac1f-df39a4a9125a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:16.401386 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cb854182-cf2e-428f-ac1f-df39a4a9125a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:16.558338 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9b55101b-ae19-4f69-89b8-d6db27c75cb3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:16.573975 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9b55101b-ae19-4f69-89b8-d6db27c75cb3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:16.575550 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9b55101b-ae19-4f69-89b8-d6db27c75cb3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:16.577776 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9b55101b-ae19-4f69-89b8-d6db27c75cb3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1750 microversion: 2.20 time: 0.241280 Jul 11 14:43:16.579539 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 223/425] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:16 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1750 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:16.672733 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-cb854182-cf2e-428f-ac1f-df39a4a9125a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:16.685010 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-cb854182-cf2e-428f-ac1f-df39a4a9125a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c" status: 200 len: 1514 microversion: 2.1 time: 0.308423 Jul 11 14:43:16.685903 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 203/426] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:16 2022] GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c => generated 1514 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:16.739572 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f03c0b2b-a19c-4a3d-8b5d-fd804ca6308c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:16.740723 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f03c0b2b-a19c-4a3d-8b5d-fd804ca6308c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:16.750833 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f03c0b2b-a19c-4a3d-8b5d-fd804ca6308c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:16.751859 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f03c0b2b-a19c-4a3d-8b5d-fd804ca6308c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:16.753433 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f03c0b2b-a19c-4a3d-8b5d-fd804ca6308c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:16.754422 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f03c0b2b-a19c-4a3d-8b5d-fd804ca6308c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:16.974775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f03c0b2b-a19c-4a3d-8b5d-fd804ca6308c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:16.986332 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f03c0b2b-a19c-4a3d-8b5d-fd804ca6308c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1367 microversion: 2.1 time: 0.250515 Jul 11 14:43:16.987594 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 224/427] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:16 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1367 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:17.599630 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d4804fcb-3817-4c17-a810-5fd7ff35f95b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:17.601645 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d4804fcb-3817-4c17-a810-5fd7ff35f95b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:17.611687 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d4804fcb-3817-4c17-a810-5fd7ff35f95b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:17.612696 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d4804fcb-3817-4c17-a810-5fd7ff35f95b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:17.613884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d4804fcb-3817-4c17-a810-5fd7ff35f95b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:17.614584 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d4804fcb-3817-4c17-a810-5fd7ff35f95b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:17.706570 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8b6a311f-fb87-473d-9937-a26a2056f504 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:17.708028 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8b6a311f-fb87-473d-9937-a26a2056f504 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:17.719119 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8b6a311f-fb87-473d-9937-a26a2056f504 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:17.719920 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8b6a311f-fb87-473d-9937-a26a2056f504 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:17.720834 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8b6a311f-fb87-473d-9937-a26a2056f504 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:17.721401 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8b6a311f-fb87-473d-9937-a26a2056f504 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:18.055354 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d4804fcb-3817-4c17-a810-5fd7ff35f95b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:18.065049 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d4804fcb-3817-4c17-a810-5fd7ff35f95b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:18.066600 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d4804fcb-3817-4c17-a810-5fd7ff35f95b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:18.067969 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d4804fcb-3817-4c17-a810-5fd7ff35f95b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1750 microversion: 2.20 time: 0.472502 Jul 11 14:43:18.079307 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 204/428] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:17 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1750 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:18.084652 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7964c0bd-8a7f-4142-97b7-fd72b80ce760 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:18.086076 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-7964c0bd-8a7f-4142-97b7-fd72b80ce760 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:18.095765 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7964c0bd-8a7f-4142-97b7-fd72b80ce760 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:18.096497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7964c0bd-8a7f-4142-97b7-fd72b80ce760 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:18.097450 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7964c0bd-8a7f-4142-97b7-fd72b80ce760 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:18.098172 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7964c0bd-8a7f-4142-97b7-fd72b80ce760 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:18.111157 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8b6a311f-fb87-473d-9937-a26a2056f504 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:18.122852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8b6a311f-fb87-473d-9937-a26a2056f504 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c" status: 200 len: 1514 microversion: 2.1 time: 0.420624 Jul 11 14:43:18.123599 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 225/429] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:17 2022] GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c => generated 1514 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:18.546066 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-d9d0368e-7828-46c7-8b19-8865b5b61c59 req-7b3b1248-4610-4d9b-8bb3-420b7af9abfb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "751f3bb7-cb4e-43c3-aab8-554e34dc828c", "name": "network-vif-deleted", "tag": "90c2c8c8-bf06-4bf9-962e-25a332ea4dba"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:43:18.567086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-d9d0368e-7828-46c7-8b19-8865b5b61c59 req-7b3b1248-4610-4d9b-8bb3-420b7af9abfb service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:18.567883 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-d9d0368e-7828-46c7-8b19-8865b5b61c59 req-7b3b1248-4610-4d9b-8bb3-420b7af9abfb service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:18.597530 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-d9d0368e-7828-46c7-8b19-8865b5b61c59 req-7b3b1248-4610-4d9b-8bb3-420b7af9abfb service nova] Creating event network-vif-deleted:90c2c8c8-bf06-4bf9-962e-25a332ea4dba for instance 751f3bb7-cb4e-43c3-aab8-554e34dc828c on hv-cinder-maste Jul 11 14:43:18.612251 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-d9d0368e-7828-46c7-8b19-8865b5b61c59 req-7b3b1248-4610-4d9b-8bb3-420b7af9abfb service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.067607 Jul 11 14:43:18.614878 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 226/430] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:43:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:43:18.670314 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7964c0bd-8a7f-4142-97b7-fd72b80ce760 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:18.680331 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-7964c0bd-8a7f-4142-97b7-fd72b80ce760 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1561 microversion: 2.1 time: 0.599564 Jul 11 14:43:18.683743 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 205/431] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:18 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1561 bytes in 604 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:18.702237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-cf72bcf6-fc54-4399-946d-f85fbf749b99 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:18.703137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-cf72bcf6-fc54-4399-946d-f85fbf749b99 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:18.713541 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cf72bcf6-fc54-4399-946d-f85fbf749b99 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:18.714255 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cf72bcf6-fc54-4399-946d-f85fbf749b99 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:18.717492 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cf72bcf6-fc54-4399-946d-f85fbf749b99 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:18.728285 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cf72bcf6-fc54-4399-946d-f85fbf749b99 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:18.856841 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-cf72bcf6-fc54-4399-946d-f85fbf749b99 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.158930 Jul 11 14:43:18.863309 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 227/432] 10.222.0.72 () {58 vars in 1274 bytes} [Mon Jul 11 14:43:18 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-interface => generated 285 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:43:19.086481 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-4f1cd225-bf66-418e-848b-51fe555f9134 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:19.087384 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-4f1cd225-bf66-418e-848b-51fe555f9134 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:19.097851 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4f1cd225-bf66-418e-848b-51fe555f9134 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:19.098320 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4f1cd225-bf66-418e-848b-51fe555f9134 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:19.099124 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4f1cd225-bf66-418e-848b-51fe555f9134 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:19.099778 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4f1cd225-bf66-418e-848b-51fe555f9134 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:19.140256 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2ccbc61f-ed06-444e-b5ec-a0145985530d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:19.141098 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2ccbc61f-ed06-444e-b5ec-a0145985530d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:19.153845 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2ccbc61f-ed06-444e-b5ec-a0145985530d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:19.154643 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2ccbc61f-ed06-444e-b5ec-a0145985530d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:19.155613 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2ccbc61f-ed06-444e-b5ec-a0145985530d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:19.156225 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2ccbc61f-ed06-444e-b5ec-a0145985530d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:19.160924 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:19.221971 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2ccbc61f-ed06-444e-b5ec-a0145985530d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:19.222722 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2ccbc61f-ed06-444e-b5ec-a0145985530d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:19.223346 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.objects.instance [None req-2ccbc61f-ed06-444e-b5ec-a0145985530d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lazy-loading 'fault' on Instance uuid 751f3bb7-cb4e-43c3-aab8-554e34dc828c {{(pid=130159) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:43:19.284563 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-2ccbc61f-ed06-444e-b5ec-a0145985530d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:19.297686 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2ccbc61f-ed06-444e-b5ec-a0145985530d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c" status: 200 len: 1309 microversion: 2.1 time: 0.161172 Jul 11 14:43:19.304520 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 228/433] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:19 2022] GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c => generated 1309 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:19.337794 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4f1cd225-bf66-418e-848b-51fe555f9134 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:19.347990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4f1cd225-bf66-418e-848b-51fe555f9134 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:19.349594 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4f1cd225-bf66-418e-848b-51fe555f9134 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:19.351279 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-4f1cd225-bf66-418e-848b-51fe555f9134 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.268376 Jul 11 14:43:19.362633 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 206/434] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:19 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:20.321565 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-7d31c707-7b7b-4add-b031-53041f2e9e73 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:20.322115 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-7d31c707-7b7b-4add-b031-53041f2e9e73 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:20.328800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7d31c707-7b7b-4add-b031-53041f2e9e73 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:20.329527 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7d31c707-7b7b-4add-b031-53041f2e9e73 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:20.331627 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7d31c707-7b7b-4add-b031-53041f2e9e73 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:20.332261 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7d31c707-7b7b-4add-b031-53041f2e9e73 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:20.371521 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e8647514-df38-4e38-a065-135165e8f9bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:20.381701 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-e8647514-df38-4e38-a065-135165e8f9bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:20.396803 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e8647514-df38-4e38-a065-135165e8f9bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:20.408469 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e8647514-df38-4e38-a065-135165e8f9bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:20.410259 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e8647514-df38-4e38-a065-135165e8f9bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:20.411219 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e8647514-df38-4e38-a065-135165e8f9bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:20.450351 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7d31c707-7b7b-4add-b031-53041f2e9e73 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:20.451551 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7d31c707-7b7b-4add-b031-53041f2e9e73 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:20.452194 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.objects.instance [None req-7d31c707-7b7b-4add-b031-53041f2e9e73 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lazy-loading 'fault' on Instance uuid 751f3bb7-cb4e-43c3-aab8-554e34dc828c {{(pid=130159) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:43:20.554244 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7d31c707-7b7b-4add-b031-53041f2e9e73 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:20.566384 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-7d31c707-7b7b-4add-b031-53041f2e9e73 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c" status: 200 len: 1309 microversion: 2.1 time: 0.249570 Jul 11 14:43:20.576175 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 229/435] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:20 2022] GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c => generated 1309 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:20.637518 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-7505bce3-23aa-40c6-8091-3846d2007cea req-083a3e5c-8007-48a6-a479-2b3826a7e2b4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f66823cd-4fe8-42cf-ab7a-d7cd7afc289a", "tag": "5a30e035-3f54-418c-8ac6-380a40fc2f5d"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:43:20.661788 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-7505bce3-23aa-40c6-8091-3846d2007cea req-083a3e5c-8007-48a6-a479-2b3826a7e2b4 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:20.662240 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-7505bce3-23aa-40c6-8091-3846d2007cea req-083a3e5c-8007-48a6-a479-2b3826a7e2b4 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:20.679711 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-7505bce3-23aa-40c6-8091-3846d2007cea req-083a3e5c-8007-48a6-a479-2b3826a7e2b4 service nova] Creating event network-changed:5a30e035-3f54-418c-8ac6-380a40fc2f5d for instance f66823cd-4fe8-42cf-ab7a-d7cd7afc289a on hv-cinder-maste Jul 11 14:43:20.697472 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-7505bce3-23aa-40c6-8091-3846d2007cea req-083a3e5c-8007-48a6-a479-2b3826a7e2b4 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063478 Jul 11 14:43:20.698401 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 230/436] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:43:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:43:20.759214 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e8647514-df38-4e38-a065-135165e8f9bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:20.770005 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e8647514-df38-4e38-a065-135165e8f9bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:20.771989 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e8647514-df38-4e38-a065-135165e8f9bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:20.773500 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e8647514-df38-4e38-a065-135165e8f9bc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.407229 Jul 11 14:43:20.774640 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 207/437] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:20 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:43:21.611639 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2c182a6a-f0ee-4894-a045-b8bd00f064e8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:21.612356 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2c182a6a-f0ee-4894-a045-b8bd00f064e8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 751f3bb7-cb4e-43c3-aab8-554e34dc828c] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:21.621356 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2c182a6a-f0ee-4894-a045-b8bd00f064e8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:21.621834 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2c182a6a-f0ee-4894-a045-b8bd00f064e8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:21.623311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2c182a6a-f0ee-4894-a045-b8bd00f064e8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:21.623760 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2c182a6a-f0ee-4894-a045-b8bd00f064e8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:21.661757 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-2c182a6a-f0ee-4894-a045-b8bd00f064e8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] HTTP exception thrown: Instance 751f3bb7-cb4e-43c3-aab8-554e34dc828c could not be found. Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2c182a6a-f0ee-4894-a045-b8bd00f064e8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Request method failure captured: Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c HTTP/1.1 Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDa2yMolE6SY6wlljmuTPJUnmChUOjyVv0PLygzOqJokF5mbLB0jheLz-pwfVGXYRs-wnAtODBfHJ5JlZwKiBzLjmpfOWcF5q6ujET7O5KmqIrZUAvGgM5IIBAgguVRVxupkMcG6kVw5ULLQzzuzLD73kI6zWOiYZg070bqMburZ8yM Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: 8e0ccc1e4683478889d90814a1305507 Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1565526209 Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:43:21.679542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:43:21.685314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/8e0ccc1e4683478889d90814a1305507"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/8e0ccc1e4683478889d90814a1305507"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_8e0ccc1e4683478889d90814a1305507"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/8e0ccc1e4683478889d90814a1305507"}]}] Jul 11 14:43:21.685314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-VolumesSnapshotTestJSON-1565526209 Jul 11 14:43:21.685314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: 8e0ccc1e4683478889d90814a1305507 Jul 11 14:43:21.685314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1565526209 Jul 11 14:43:21.685314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-VolumesSnapshotTestJSON-1565526209-project Jul 11 14:43:21.685314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:43:21.685314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:43:21.685314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: b7457a38a0594e8da4c053a5c6a4aa15 Jul 11 14:43:21.685314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-VolumesSnapshotTestJSON-1565526209-project Jul 11 14:43:21.685314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:43:21.685314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance 751f3bb7-cb4e-43c3-aab8-554e34dc828c could not be found. Jul 11 14:43:21.685314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '751f3bb7-cb4e-43c3-aab8-554e34dc828c'} Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 751f3bb7-cb4e-43c3-aab8-554e34dc828c could not be found. Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:43:21.686925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:43:21.688415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:43:21.688415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:43:21.688415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:43:21.688415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:43:21.688415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:43:21.688415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:43:21.688415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:43:21.688415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 751f3bb7-cb4e-43c3-aab8-554e34dc828c could not be found. Jul 11 14:43:21.688415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:43:21.688415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:43:21.688415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:43:21.688415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:43:21.688415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:43:21.688415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:43:21.688415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:43:21.688415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:43:21.688415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 751f3bb7-cb4e-43c3-aab8-554e34dc828c could not be found. Jul 11 14:43:21.688415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:43:21.690746 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2c182a6a-f0ee-4894-a045-b8bd00f064e8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Returning 404 to user: Instance 751f3bb7-cb4e-43c3-aab8-554e34dc828c could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:43:21.691950 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2c182a6a-f0ee-4894-a045-b8bd00f064e8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c" status: 404 len: 111 microversion: 2.1 time: 0.098932 Jul 11 14:43:21.695877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 231/438] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:21 2022] GET /compute/v2.1/servers/751f3bb7-cb4e-43c3-aab8-554e34dc828c => generated 111 bytes in 101 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 11 14:43:21.793960 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8d81a945-e71b-4643-ab39-c7d7a3e1c487 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:21.795243 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-8d81a945-e71b-4643-ab39-c7d7a3e1c487 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:21.807805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d81a945-e71b-4643-ab39-c7d7a3e1c487 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:21.808473 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d81a945-e71b-4643-ab39-c7d7a3e1c487 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:21.809429 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d81a945-e71b-4643-ab39-c7d7a3e1c487 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:21.810017 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d81a945-e71b-4643-ab39-c7d7a3e1c487 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:21.998337 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8d81a945-e71b-4643-ab39-c7d7a3e1c487 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:22.007540 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8d81a945-e71b-4643-ab39-c7d7a3e1c487 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:22.010024 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8d81a945-e71b-4643-ab39-c7d7a3e1c487 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:22.011252 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8d81a945-e71b-4643-ab39-c7d7a3e1c487 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.221847 Jul 11 14:43:22.012102 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 208/439] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:21 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:22.658032 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c999d9ef-0854-4206-a05b-b804166cceba tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Action: 'create', calling method: .version_select at 0x7f593010a8b0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-749000618", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDqSP6aw/ZYxjs/hqNyCrp1Nh+LR8hUAIRF8U2B/IJZl9lh9/I+bvLWzTszBHvt1OrXtgWNYot3MqoDfNCagpeENmR0ow4a4UCzdPw08aRSnqqOsw+P3eAlQx30g7PF9iQ=="}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:43:22.678196 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-c999d9ef-0854-4206-a05b-b804166cceba tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Getting quotas for project 3e77cfd4544c45468c57510de6034e38. Resources: {'key_pairs'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:43:22.682915 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-c999d9ef-0854-4206-a05b-b804166cceba tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Getting quotas for user 82025bd919ce404b96491a19262d4281 and project 3e77cfd4544c45468c57510de6034e38. Resources: {'key_pairs'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:43:22.713659 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c999d9ef-0854-4206-a05b-b804166cceba tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.059538 Jul 11 14:43:22.714649 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 232/440] 10.222.0.72 () {60 vars in 1158 bytes} [Mon Jul 11 14:43:22 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:43:22.783210 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-98672b22-c85f-4809-ba77-1fd94d1929be tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-380104456", "imageRef": "cf3dbf8c-39f2-44c4-bb10-4d89684c7f59", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-749000618", "networks": [{"uuid": "06a4c46d-d89a-4730-8741-8893e973bf68"}]}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:43:22.854346 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.network.neutron [None req-98672b22-c85f-4809-ba77-1fd94d1929be tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] validate_networks() for [('06a4c46d-d89a-4730-8741-8893e973bf68', None, None, None, None, None)] {{(pid=130160) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:43:23.032316 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-59f7185d-7cb6-4a7a-aac7-30fef840d06b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:23.034730 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-59f7185d-7cb6-4a7a-aac7-30fef840d06b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:23.047789 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-59f7185d-7cb6-4a7a-aac7-30fef840d06b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:23.048599 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-59f7185d-7cb6-4a7a-aac7-30fef840d06b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:23.049621 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-59f7185d-7cb6-4a7a-aac7-30fef840d06b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:23.050280 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-59f7185d-7cb6-4a7a-aac7-30fef840d06b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:23.238010 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-59f7185d-7cb6-4a7a-aac7-30fef840d06b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:23.247268 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-59f7185d-7cb6-4a7a-aac7-30fef840d06b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:23.248775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-59f7185d-7cb6-4a7a-aac7-30fef840d06b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:23.249931 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-59f7185d-7cb6-4a7a-aac7-30fef840d06b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.224544 Jul 11 14:43:23.250503 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 233/441] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:23 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:23.332838 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-98672b22-c85f-4809-ba77-1fd94d1929be tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Flavor limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:43:23.333262 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-98672b22-c85f-4809-ba77-1fd94d1929be tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Image limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:43:23.333736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-98672b22-c85f-4809-ba77-1fd94d1929be tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Flavor pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:43:23.334147 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-98672b22-c85f-4809-ba77-1fd94d1929be tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Image pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:43:23.334643 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-98672b22-c85f-4809-ba77-1fd94d1929be tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:43:23.343497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-98672b22-c85f-4809-ba77-1fd94d1929be tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:23.343940 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-98672b22-c85f-4809-ba77-1fd94d1929be tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:23.344760 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-98672b22-c85f-4809-ba77-1fd94d1929be tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:23.345194 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-98672b22-c85f-4809-ba77-1fd94d1929be tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:23.363028 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-98672b22-c85f-4809-ba77-1fd94d1929be tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Getting quotas for project 3e77cfd4544c45468c57510de6034e38. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:43:23.367928 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-98672b22-c85f-4809-ba77-1fd94d1929be tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Getting quotas for user 82025bd919ce404b96491a19262d4281 and project 3e77cfd4544c45468c57510de6034e38. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:43:23.380905 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-98672b22-c85f-4809-ba77-1fd94d1929be tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Going to run 1 instances... {{(pid=130160) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:43:23.383383 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-98672b22-c85f-4809-ba77-1fd94d1929be tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='cf3dbf8c-39f2-44c4-bb10-4d89684c7f59',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130160) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:43:23.442076 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-98672b22-c85f-4809-ba77-1fd94d1929be tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.662645 Jul 11 14:43:23.443790 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 209/442] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:43:22 2022] POST /compute/v2.1/servers => generated 380 bytes in 666 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 11 14:43:23.456990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-710a0431-a4e3-4787-98e1-8d24f4752ada tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:23.467427 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-710a0431-a4e3-4787-98e1-8d24f4752ada tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:23.524602 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-710a0431-a4e3-4787-98e1-8d24f4752ada tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:23.532203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-710a0431-a4e3-4787-98e1-8d24f4752ada tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1257 microversion: 2.1 time: 0.082798 Jul 11 14:43:23.532841 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 234/443] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:23 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1257 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:24.268725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-6f3eb265-f5f4-4b4e-ad63-20f371ba56f7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:24.269457 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-6f3eb265-f5f4-4b4e-ad63-20f371ba56f7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:24.276973 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6f3eb265-f5f4-4b4e-ad63-20f371ba56f7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:24.277748 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6f3eb265-f5f4-4b4e-ad63-20f371ba56f7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:24.278885 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6f3eb265-f5f4-4b4e-ad63-20f371ba56f7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:24.279531 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6f3eb265-f5f4-4b4e-ad63-20f371ba56f7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:24.418213 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6f3eb265-f5f4-4b4e-ad63-20f371ba56f7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:24.428996 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6f3eb265-f5f4-4b4e-ad63-20f371ba56f7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:24.430571 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6f3eb265-f5f4-4b4e-ad63-20f371ba56f7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:24.431742 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-6f3eb265-f5f4-4b4e-ad63-20f371ba56f7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.167629 Jul 11 14:43:24.432623 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 210/444] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:24 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:24.550015 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-7bdc77df-1c90-4884-a6ea-5167e0c5ca9e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:24.550679 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-7bdc77df-1c90-4884-a6ea-5167e0c5ca9e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:24.557853 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7bdc77df-1c90-4884-a6ea-5167e0c5ca9e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:24.558308 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7bdc77df-1c90-4884-a6ea-5167e0c5ca9e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:24.559109 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7bdc77df-1c90-4884-a6ea-5167e0c5ca9e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:24.559553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7bdc77df-1c90-4884-a6ea-5167e0c5ca9e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:24.624266 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7bdc77df-1c90-4884-a6ea-5167e0c5ca9e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:24.631837 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-7bdc77df-1c90-4884-a6ea-5167e0c5ca9e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1261 microversion: 2.1 time: 0.085688 Jul 11 14:43:24.632391 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 235/445] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:24 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1261 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:25.450932 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-1fbd1f8a-5aee-469d-bb0e-e8d57c6142d1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:25.451950 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-1fbd1f8a-5aee-469d-bb0e-e8d57c6142d1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:25.459545 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1fbd1f8a-5aee-469d-bb0e-e8d57c6142d1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:25.459984 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1fbd1f8a-5aee-469d-bb0e-e8d57c6142d1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:25.460764 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1fbd1f8a-5aee-469d-bb0e-e8d57c6142d1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:25.461206 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1fbd1f8a-5aee-469d-bb0e-e8d57c6142d1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:25.603171 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-1fbd1f8a-5aee-469d-bb0e-e8d57c6142d1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:25.612567 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-1fbd1f8a-5aee-469d-bb0e-e8d57c6142d1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:25.614389 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-1fbd1f8a-5aee-469d-bb0e-e8d57c6142d1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:25.615850 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-1fbd1f8a-5aee-469d-bb0e-e8d57c6142d1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.169339 Jul 11 14:43:25.626470 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 211/446] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:25 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:25.650574 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-97c7ba04-5423-48a8-8168-d24f564106ee tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:25.651148 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-97c7ba04-5423-48a8-8168-d24f564106ee tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:25.657901 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-97c7ba04-5423-48a8-8168-d24f564106ee tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:25.658360 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-97c7ba04-5423-48a8-8168-d24f564106ee tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:25.659179 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-97c7ba04-5423-48a8-8168-d24f564106ee tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:25.659621 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-97c7ba04-5423-48a8-8168-d24f564106ee tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:25.733845 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-97c7ba04-5423-48a8-8168-d24f564106ee tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:25.742253 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-97c7ba04-5423-48a8-8168-d24f564106ee tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1261 microversion: 2.1 time: 0.096272 Jul 11 14:43:25.742957 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 236/447] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:25 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1261 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:26.635273 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-5d514378-86a0-42c0-87e2-186c2b8147b2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:26.635932 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-5d514378-86a0-42c0-87e2-186c2b8147b2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:26.642938 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5d514378-86a0-42c0-87e2-186c2b8147b2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:26.642938 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5d514378-86a0-42c0-87e2-186c2b8147b2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:26.643576 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5d514378-86a0-42c0-87e2-186c2b8147b2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:26.644017 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5d514378-86a0-42c0-87e2-186c2b8147b2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:26.760048 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a85af80c-ee15-49c7-bb6c-d1c85a1602d0 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:26.760639 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a85af80c-ee15-49c7-bb6c-d1c85a1602d0 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:26.780586 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a85af80c-ee15-49c7-bb6c-d1c85a1602d0 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:26.780973 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5d514378-86a0-42c0-87e2-186c2b8147b2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:26.781103 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a85af80c-ee15-49c7-bb6c-d1c85a1602d0 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:26.781834 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a85af80c-ee15-49c7-bb6c-d1c85a1602d0 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:26.782271 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a85af80c-ee15-49c7-bb6c-d1c85a1602d0 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:26.792897 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5d514378-86a0-42c0-87e2-186c2b8147b2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:26.794482 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5d514378-86a0-42c0-87e2-186c2b8147b2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:26.795819 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-5d514378-86a0-42c0-87e2-186c2b8147b2 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.165114 Jul 11 14:43:26.796518 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 212/448] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:26 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:26.847733 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a85af80c-ee15-49c7-bb6c-d1c85a1602d0 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:26.856664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a85af80c-ee15-49c7-bb6c-d1c85a1602d0 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1261 microversion: 2.1 time: 0.100883 Jul 11 14:43:26.857707 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 237/449] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:26 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1261 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:27.820461 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-494ec5dd-7021-4247-8d1f-2c6861ef7e8e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:27.822347 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-494ec5dd-7021-4247-8d1f-2c6861ef7e8e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:27.836714 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-494ec5dd-7021-4247-8d1f-2c6861ef7e8e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:27.838011 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-494ec5dd-7021-4247-8d1f-2c6861ef7e8e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:27.839027 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-494ec5dd-7021-4247-8d1f-2c6861ef7e8e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:27.839628 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-494ec5dd-7021-4247-8d1f-2c6861ef7e8e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:27.875894 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-73029981-5eb1-4f26-b3c5-14d3adce169e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:27.876528 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-73029981-5eb1-4f26-b3c5-14d3adce169e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:27.885126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-73029981-5eb1-4f26-b3c5-14d3adce169e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:27.885589 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-73029981-5eb1-4f26-b3c5-14d3adce169e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:27.886356 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-73029981-5eb1-4f26-b3c5-14d3adce169e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:27.886826 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-73029981-5eb1-4f26-b3c5-14d3adce169e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:27.956278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-73029981-5eb1-4f26-b3c5-14d3adce169e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:27.965969 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-73029981-5eb1-4f26-b3c5-14d3adce169e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1261 microversion: 2.1 time: 0.093861 Jul 11 14:43:27.966721 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 238/450] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:27 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1261 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:28.008521 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-494ec5dd-7021-4247-8d1f-2c6861ef7e8e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:28.022790 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-494ec5dd-7021-4247-8d1f-2c6861ef7e8e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:28.024899 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-494ec5dd-7021-4247-8d1f-2c6861ef7e8e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:28.026266 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-494ec5dd-7021-4247-8d1f-2c6861ef7e8e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.214710 Jul 11 14:43:28.027085 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 213/451] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:27 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:28.984696 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-570895ee-d8e0-4e1b-afaa-1fbb8c9e2c35 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:28.985302 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-570895ee-d8e0-4e1b-afaa-1fbb8c9e2c35 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:28.991541 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-570895ee-d8e0-4e1b-afaa-1fbb8c9e2c35 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:28.991991 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-570895ee-d8e0-4e1b-afaa-1fbb8c9e2c35 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:28.992754 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-570895ee-d8e0-4e1b-afaa-1fbb8c9e2c35 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:28.993184 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-570895ee-d8e0-4e1b-afaa-1fbb8c9e2c35 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:29.045508 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-135c74d7-b578-49c1-84ec-960831f7cbe4 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:29.046976 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-135c74d7-b578-49c1-84ec-960831f7cbe4 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:29.057196 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-135c74d7-b578-49c1-84ec-960831f7cbe4 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:29.059041 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-135c74d7-b578-49c1-84ec-960831f7cbe4 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:29.059944 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-135c74d7-b578-49c1-84ec-960831f7cbe4 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:29.060449 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-135c74d7-b578-49c1-84ec-960831f7cbe4 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:29.076448 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-570895ee-d8e0-4e1b-afaa-1fbb8c9e2c35 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:29.085871 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-570895ee-d8e0-4e1b-afaa-1fbb8c9e2c35 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1261 microversion: 2.1 time: 0.105047 Jul 11 14:43:29.086807 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 239/452] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:28 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1261 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:29.213965 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-135c74d7-b578-49c1-84ec-960831f7cbe4 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:29.223773 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-135c74d7-b578-49c1-84ec-960831f7cbe4 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:29.225473 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-135c74d7-b578-49c1-84ec-960831f7cbe4 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:29.227603 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-135c74d7-b578-49c1-84ec-960831f7cbe4 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.186197 Jul 11 14:43:29.228846 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 214/453] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:29 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:30.103902 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1d93ab64-5a9e-4705-9cb2-ae6b5ef24617 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:30.104753 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1d93ab64-5a9e-4705-9cb2-ae6b5ef24617 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:30.111603 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1d93ab64-5a9e-4705-9cb2-ae6b5ef24617 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:30.112053 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1d93ab64-5a9e-4705-9cb2-ae6b5ef24617 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:30.112819 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1d93ab64-5a9e-4705-9cb2-ae6b5ef24617 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:30.113415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1d93ab64-5a9e-4705-9cb2-ae6b5ef24617 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:30.183385 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1d93ab64-5a9e-4705-9cb2-ae6b5ef24617 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:30.191281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1d93ab64-5a9e-4705-9cb2-ae6b5ef24617 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1261 microversion: 2.1 time: 0.091871 Jul 11 14:43:30.192059 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 240/454] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:30 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1261 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:30.249511 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-56e4b5fd-5d00-421a-87ea-1fc08d79d423 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:30.251033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-56e4b5fd-5d00-421a-87ea-1fc08d79d423 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:30.258295 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-56e4b5fd-5d00-421a-87ea-1fc08d79d423 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:30.259140 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-56e4b5fd-5d00-421a-87ea-1fc08d79d423 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:30.260153 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-56e4b5fd-5d00-421a-87ea-1fc08d79d423 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:30.260950 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-56e4b5fd-5d00-421a-87ea-1fc08d79d423 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:30.404046 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-56e4b5fd-5d00-421a-87ea-1fc08d79d423 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:30.417137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-56e4b5fd-5d00-421a-87ea-1fc08d79d423 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:30.419317 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-56e4b5fd-5d00-421a-87ea-1fc08d79d423 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:30.420524 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-56e4b5fd-5d00-421a-87ea-1fc08d79d423 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.178030 Jul 11 14:43:30.429725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 215/455] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:30 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:31.208632 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8535b184-ba83-478c-87a3-85aa2e33d86e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:31.209386 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8535b184-ba83-478c-87a3-85aa2e33d86e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:31.215978 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8535b184-ba83-478c-87a3-85aa2e33d86e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:31.216427 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8535b184-ba83-478c-87a3-85aa2e33d86e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:31.217261 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8535b184-ba83-478c-87a3-85aa2e33d86e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:31.217705 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8535b184-ba83-478c-87a3-85aa2e33d86e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:31.279488 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8535b184-ba83-478c-87a3-85aa2e33d86e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:31.287134 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8535b184-ba83-478c-87a3-85aa2e33d86e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1261 microversion: 2.1 time: 0.082668 Jul 11 14:43:31.287775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 241/456] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:31 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1261 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:31.439511 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-6a3638c6-d7c6-4b6b-b309-eef1dd112ce7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:31.440530 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-6a3638c6-d7c6-4b6b-b309-eef1dd112ce7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:31.447412 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6a3638c6-d7c6-4b6b-b309-eef1dd112ce7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:31.447859 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6a3638c6-d7c6-4b6b-b309-eef1dd112ce7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:31.448616 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6a3638c6-d7c6-4b6b-b309-eef1dd112ce7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:31.449050 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6a3638c6-d7c6-4b6b-b309-eef1dd112ce7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:31.589984 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6a3638c6-d7c6-4b6b-b309-eef1dd112ce7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:31.601612 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6a3638c6-d7c6-4b6b-b309-eef1dd112ce7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:31.604083 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6a3638c6-d7c6-4b6b-b309-eef1dd112ce7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:31.605299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-6a3638c6-d7c6-4b6b-b309-eef1dd112ce7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.169840 Jul 11 14:43:31.605860 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 216/457] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:31 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:31.827918 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-0c7c58ed-e9b4-4d3a-b99d-7bf603ceec04 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-107779211", "imageRef": "a588fa24-23b5-4e53-87c6-7e600ee3a5ef", "flavorRef": "452", "networks": [{"uuid": "c0778f11-32ed-4b63-9a0c-1270df9f375b"}]}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:43:31.885351 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.network.neutron [None req-0c7c58ed-e9b4-4d3a-b99d-7bf603ceec04 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] validate_networks() for [('c0778f11-32ed-4b63-9a0c-1270df9f375b', None, None, None, None, None)] {{(pid=130159) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:43:32.241922 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-0c7c58ed-e9b4-4d3a-b99d-7bf603ceec04 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Flavor limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:43:32.242864 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-0c7c58ed-e9b4-4d3a-b99d-7bf603ceec04 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Image limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:43:32.243021 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-0c7c58ed-e9b4-4d3a-b99d-7bf603ceec04 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Flavor pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:43:32.243432 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-0c7c58ed-e9b4-4d3a-b99d-7bf603ceec04 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Image pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:43:32.243976 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-0c7c58ed-e9b4-4d3a-b99d-7bf603ceec04 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:43:32.255130 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0c7c58ed-e9b4-4d3a-b99d-7bf603ceec04 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:32.255582 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0c7c58ed-e9b4-4d3a-b99d-7bf603ceec04 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:32.256377 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0c7c58ed-e9b4-4d3a-b99d-7bf603ceec04 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:32.256876 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0c7c58ed-e9b4-4d3a-b99d-7bf603ceec04 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:32.274783 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-0c7c58ed-e9b4-4d3a-b99d-7bf603ceec04 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Getting quotas for project 8e0ccc1e4683478889d90814a1305507. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:43:32.278848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-0c7c58ed-e9b4-4d3a-b99d-7bf603ceec04 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Getting quotas for user b7457a38a0594e8da4c053a5c6a4aa15 and project 8e0ccc1e4683478889d90814a1305507. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:43:32.291041 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-0c7c58ed-e9b4-4d3a-b99d-7bf603ceec04 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Going to run 1 instances... {{(pid=130159) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:43:32.293292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-0c7c58ed-e9b4-4d3a-b99d-7bf603ceec04 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a588fa24-23b5-4e53-87c6-7e600ee3a5ef',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130159) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:43:32.305295 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-6d9bd024-7b46-4c0b-b926-7df678bc8156 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:32.306088 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-6d9bd024-7b46-4c0b-b926-7df678bc8156 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:32.312307 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6d9bd024-7b46-4c0b-b926-7df678bc8156 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:32.312850 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6d9bd024-7b46-4c0b-b926-7df678bc8156 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:32.313736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6d9bd024-7b46-4c0b-b926-7df678bc8156 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:32.314237 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6d9bd024-7b46-4c0b-b926-7df678bc8156 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:32.314980 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=96,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:32.345987 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-0c7c58ed-e9b4-4d3a-b99d-7bf603ceec04 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.522359 Jul 11 14:43:32.346841 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 242/458] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:43:31 2022] POST /compute/v2.1/servers => generated 380 bytes in 524 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 11 14:43:32.356874 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2cfed8ed-6044-41c6-9b65-856d62769956 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:32.357697 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2cfed8ed-6044-41c6-9b65-856d62769956 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:32.396867 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6d9bd024-7b46-4c0b-b926-7df678bc8156 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:32.409630 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-6d9bd024-7b46-4c0b-b926-7df678bc8156 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1261 microversion: 2.1 time: 0.108328 Jul 11 14:43:32.420276 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 217/459] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:32 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1261 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:32.421813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-2cfed8ed-6044-41c6-9b65-856d62769956 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:32.429064 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2cfed8ed-6044-41c6-9b65-856d62769956 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1218 microversion: 2.1 time: 0.076294 Jul 11 14:43:32.429616 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 243/460] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:32 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1218 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:32.623725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-76330b01-2d89-47c5-83a3-53f0fedcce3e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:32.624356 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-76330b01-2d89-47c5-83a3-53f0fedcce3e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:32.630576 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-76330b01-2d89-47c5-83a3-53f0fedcce3e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:32.631032 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-76330b01-2d89-47c5-83a3-53f0fedcce3e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:32.631810 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-76330b01-2d89-47c5-83a3-53f0fedcce3e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:32.632252 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-76330b01-2d89-47c5-83a3-53f0fedcce3e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:32.778913 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-76330b01-2d89-47c5-83a3-53f0fedcce3e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:32.781124 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=100 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:32.789893 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-76330b01-2d89-47c5-83a3-53f0fedcce3e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:32.791457 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-76330b01-2d89-47c5-83a3-53f0fedcce3e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:32.792672 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-76330b01-2d89-47c5-83a3-53f0fedcce3e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.172870 Jul 11 14:43:32.793228 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 218/461] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:32 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:33.435963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-10eec542-f016-4a28-991e-b5f91ebf4dd3 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:33.436586 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-10eec542-f016-4a28-991e-b5f91ebf4dd3 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:33.444400 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-10eec542-f016-4a28-991e-b5f91ebf4dd3 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:33.444871 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-10eec542-f016-4a28-991e-b5f91ebf4dd3 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:33.445634 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-10eec542-f016-4a28-991e-b5f91ebf4dd3 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:33.447125 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-10eec542-f016-4a28-991e-b5f91ebf4dd3 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:33.448288 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-85793bfd-ceab-49fd-bb3c-99014efb7caf tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:33.448909 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-85793bfd-ceab-49fd-bb3c-99014efb7caf tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:33.451960 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=2 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:33.452742 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=6 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:33.456840 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-85793bfd-ceab-49fd-bb3c-99014efb7caf tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:33.457289 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-85793bfd-ceab-49fd-bb3c-99014efb7caf tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:33.458309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-85793bfd-ceab-49fd-bb3c-99014efb7caf tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:33.458771 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-85793bfd-ceab-49fd-bb3c-99014efb7caf tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:33.535927 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-85793bfd-ceab-49fd-bb3c-99014efb7caf tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:33.544321 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-85793bfd-ceab-49fd-bb3c-99014efb7caf tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1214 microversion: 2.1 time: 0.101257 Jul 11 14:43:33.545936 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 219/462] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:33 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1214 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:33.547446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-10eec542-f016-4a28-991e-b5f91ebf4dd3 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:33.555159 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-10eec542-f016-4a28-991e-b5f91ebf4dd3 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1253 microversion: 2.1 time: 0.123422 Jul 11 14:43:33.556250 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 244/463] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:33 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1253 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:33.811601 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a37a3be0-2df0-4a26-917f-b3c80540e766 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:33.812220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a37a3be0-2df0-4a26-917f-b3c80540e766 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:33.818890 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a37a3be0-2df0-4a26-917f-b3c80540e766 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:33.819373 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a37a3be0-2df0-4a26-917f-b3c80540e766 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:33.820105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a37a3be0-2df0-4a26-917f-b3c80540e766 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:33.820549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a37a3be0-2df0-4a26-917f-b3c80540e766 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:33.956304 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a37a3be0-2df0-4a26-917f-b3c80540e766 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:33.965198 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a37a3be0-2df0-4a26-917f-b3c80540e766 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:33.966749 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a37a3be0-2df0-4a26-917f-b3c80540e766 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:33.967886 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a37a3be0-2df0-4a26-917f-b3c80540e766 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.160376 Jul 11 14:43:33.968438 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 220/464] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:33 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:34.562840 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-866f7374-2b47-491f-8431-406a77431b4a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:34.564375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-866f7374-2b47-491f-8431-406a77431b4a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:34.574592 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-866f7374-2b47-491f-8431-406a77431b4a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:34.575704 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-866f7374-2b47-491f-8431-406a77431b4a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:34.577115 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-866f7374-2b47-491f-8431-406a77431b4a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:34.577734 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7811be9c-b8af-463d-bfb5-45aa7f719c7c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:34.577919 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-866f7374-2b47-491f-8431-406a77431b4a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:34.578613 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-7811be9c-b8af-463d-bfb5-45aa7f719c7c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:34.586087 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7811be9c-b8af-463d-bfb5-45aa7f719c7c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:34.586746 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7811be9c-b8af-463d-bfb5-45aa7f719c7c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:34.587635 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7811be9c-b8af-463d-bfb5-45aa7f719c7c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:34.588147 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7811be9c-b8af-463d-bfb5-45aa7f719c7c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:34.722662 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7811be9c-b8af-463d-bfb5-45aa7f719c7c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:34.727328 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-866f7374-2b47-491f-8431-406a77431b4a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:34.733140 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-7811be9c-b8af-463d-bfb5-45aa7f719c7c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1315 microversion: 2.1 time: 0.162421 Jul 11 14:43:34.734052 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 221/465] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:34 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1315 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:34.752986 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-866f7374-2b47-491f-8431-406a77431b4a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1276 microversion: 2.1 time: 0.193827 Jul 11 14:43:34.754839 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 245/466] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:34 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1276 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:34.996992 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-89b94dae-2b36-4424-9792-1f2d0c41fc27 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:34.997901 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-89b94dae-2b36-4424-9792-1f2d0c41fc27 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:35.011713 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-89b94dae-2b36-4424-9792-1f2d0c41fc27 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:35.013600 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-89b94dae-2b36-4424-9792-1f2d0c41fc27 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:35.015665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-89b94dae-2b36-4424-9792-1f2d0c41fc27 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:35.018107 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-89b94dae-2b36-4424-9792-1f2d0c41fc27 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:35.363579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-89b94dae-2b36-4424-9792-1f2d0c41fc27 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:35.386009 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-89b94dae-2b36-4424-9792-1f2d0c41fc27 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:35.402097 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-89b94dae-2b36-4424-9792-1f2d0c41fc27 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:35.409949 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-89b94dae-2b36-4424-9792-1f2d0c41fc27 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.421868 Jul 11 14:43:35.410959 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 222/467] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:34 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:35.754149 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-02f2c411-6f32-4745-88c9-1a008a29fe58 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:35.768627 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-02f2c411-6f32-4745-88c9-1a008a29fe58 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:35.775248 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c22f35f8-e409-4af8-b2bd-6a3557a8c2ad tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:35.775890 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c22f35f8-e409-4af8-b2bd-6a3557a8c2ad tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:35.781091 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-02f2c411-6f32-4745-88c9-1a008a29fe58 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:35.781640 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-02f2c411-6f32-4745-88c9-1a008a29fe58 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:35.783658 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c22f35f8-e409-4af8-b2bd-6a3557a8c2ad tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:35.784508 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c22f35f8-e409-4af8-b2bd-6a3557a8c2ad tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:35.784813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-02f2c411-6f32-4745-88c9-1a008a29fe58 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:35.785521 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-02f2c411-6f32-4745-88c9-1a008a29fe58 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:35.786178 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c22f35f8-e409-4af8-b2bd-6a3557a8c2ad tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:35.787072 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c22f35f8-e409-4af8-b2bd-6a3557a8c2ad tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:35.957336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c22f35f8-e409-4af8-b2bd-6a3557a8c2ad tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:35.962190 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-02f2c411-6f32-4745-88c9-1a008a29fe58 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:35.965624 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c22f35f8-e409-4af8-b2bd-6a3557a8c2ad tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1318 microversion: 2.1 time: 0.195298 Jul 11 14:43:35.966216 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 223/468] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:35 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1318 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:35.970164 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-02f2c411-6f32-4745-88c9-1a008a29fe58 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.220671 Jul 11 14:43:35.974811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 246/469] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:35 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:36.445182 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c28db564-6e8a-4239-98c8-e0b4685644c6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:36.445824 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c28db564-6e8a-4239-98c8-e0b4685644c6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:36.453811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c28db564-6e8a-4239-98c8-e0b4685644c6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:36.454277 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c28db564-6e8a-4239-98c8-e0b4685644c6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:36.455112 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c28db564-6e8a-4239-98c8-e0b4685644c6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:36.455552 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c28db564-6e8a-4239-98c8-e0b4685644c6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:36.603773 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c28db564-6e8a-4239-98c8-e0b4685644c6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:36.613477 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c28db564-6e8a-4239-98c8-e0b4685644c6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:36.615029 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c28db564-6e8a-4239-98c8-e0b4685644c6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:36.616203 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c28db564-6e8a-4239-98c8-e0b4685644c6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.175003 Jul 11 14:43:36.616960 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 224/470] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:36 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:43:36.983545 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9e9ed172-d5d7-4e18-838a-0d2bb0c032e1 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:36.984211 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9e9ed172-d5d7-4e18-838a-0d2bb0c032e1 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:36.986872 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-65443492-eceb-4d7f-9b73-cffc794fab5b tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:36.987496 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-65443492-eceb-4d7f-9b73-cffc794fab5b tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:36.991566 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9e9ed172-d5d7-4e18-838a-0d2bb0c032e1 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:36.992313 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9e9ed172-d5d7-4e18-838a-0d2bb0c032e1 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:36.994802 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-65443492-eceb-4d7f-9b73-cffc794fab5b tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:36.995266 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-65443492-eceb-4d7f-9b73-cffc794fab5b tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:36.996024 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-65443492-eceb-4d7f-9b73-cffc794fab5b tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:36.996342 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9e9ed172-d5d7-4e18-838a-0d2bb0c032e1 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:36.996455 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-65443492-eceb-4d7f-9b73-cffc794fab5b tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:36.996944 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9e9ed172-d5d7-4e18-838a-0d2bb0c032e1 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:37.146665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-65443492-eceb-4d7f-9b73-cffc794fab5b tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:37.151374 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9e9ed172-d5d7-4e18-838a-0d2bb0c032e1 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:37.154557 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-65443492-eceb-4d7f-9b73-cffc794fab5b tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.171279 Jul 11 14:43:37.155365 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 225/471] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:36 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:37.159632 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9e9ed172-d5d7-4e18-838a-0d2bb0c032e1 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1318 microversion: 2.1 time: 0.180368 Jul 11 14:43:37.160847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 247/472] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:36 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1318 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:37.635017 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9e2a511d-8f40-419a-a3df-e0e64959fe7e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:37.635629 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-9e2a511d-8f40-419a-a3df-e0e64959fe7e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:37.642398 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9e2a511d-8f40-419a-a3df-e0e64959fe7e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:37.642953 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9e2a511d-8f40-419a-a3df-e0e64959fe7e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:37.643900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9e2a511d-8f40-419a-a3df-e0e64959fe7e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:37.644159 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9e2a511d-8f40-419a-a3df-e0e64959fe7e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:37.813327 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-9e2a511d-8f40-419a-a3df-e0e64959fe7e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:37.822973 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-9e2a511d-8f40-419a-a3df-e0e64959fe7e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:37.824492 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-9e2a511d-8f40-419a-a3df-e0e64959fe7e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:37.825653 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9e2a511d-8f40-419a-a3df-e0e64959fe7e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.195172 Jul 11 14:43:37.826210 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 226/473] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:37 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:38.175815 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-481d90fb-6f22-4470-bf22-96d6bbda9df4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:38.176556 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-481d90fb-6f22-4470-bf22-96d6bbda9df4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:38.176690 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d619ee15-c33d-491d-b1c5-e232acd32942 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:38.177302 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d619ee15-c33d-491d-b1c5-e232acd32942 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:38.184077 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-481d90fb-6f22-4470-bf22-96d6bbda9df4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:38.184650 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d619ee15-c33d-491d-b1c5-e232acd32942 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:38.184827 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-481d90fb-6f22-4470-bf22-96d6bbda9df4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:38.185191 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d619ee15-c33d-491d-b1c5-e232acd32942 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:38.185833 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-481d90fb-6f22-4470-bf22-96d6bbda9df4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:38.186215 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d619ee15-c33d-491d-b1c5-e232acd32942 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:38.186339 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-481d90fb-6f22-4470-bf22-96d6bbda9df4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:38.192309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d619ee15-c33d-491d-b1c5-e232acd32942 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:38.323359 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-481d90fb-6f22-4470-bf22-96d6bbda9df4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:38.332529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-481d90fb-6f22-4470-bf22-96d6bbda9df4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.160647 Jul 11 14:43:38.333795 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 248/474] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:38 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:38.354965 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d619ee15-c33d-491d-b1c5-e232acd32942 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:38.362526 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d619ee15-c33d-491d-b1c5-e232acd32942 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1318 microversion: 2.1 time: 0.189488 Jul 11 14:43:38.363100 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 227/475] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:38 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1318 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:38.844510 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-6ee9f139-f955-45c9-9c7a-e7aea08ae3eb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:38.845133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-6ee9f139-f955-45c9-9c7a-e7aea08ae3eb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:38.851900 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6ee9f139-f955-45c9-9c7a-e7aea08ae3eb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:38.852851 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6ee9f139-f955-45c9-9c7a-e7aea08ae3eb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:38.853632 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6ee9f139-f955-45c9-9c7a-e7aea08ae3eb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:38.854078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6ee9f139-f955-45c9-9c7a-e7aea08ae3eb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:38.858522 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=100 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:38.995532 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-6ee9f139-f955-45c9-9c7a-e7aea08ae3eb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:39.004598 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-6ee9f139-f955-45c9-9c7a-e7aea08ae3eb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:39.006128 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-6ee9f139-f955-45c9-9c7a-e7aea08ae3eb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:39.007345 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-6ee9f139-f955-45c9-9c7a-e7aea08ae3eb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.166805 Jul 11 14:43:39.007912 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 249/476] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:38 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:39.351445 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-64186307-4574-4c96-ad0d-73ffa7c91f22 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:39.352095 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-64186307-4574-4c96-ad0d-73ffa7c91f22 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:39.358827 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-64186307-4574-4c96-ad0d-73ffa7c91f22 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:39.360674 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-64186307-4574-4c96-ad0d-73ffa7c91f22 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:39.361600 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-64186307-4574-4c96-ad0d-73ffa7c91f22 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:39.362121 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-64186307-4574-4c96-ad0d-73ffa7c91f22 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:39.381881 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f042d149-86a2-45d2-a75a-301f77712b8c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:39.383440 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f042d149-86a2-45d2-a75a-301f77712b8c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:39.390878 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f042d149-86a2-45d2-a75a-301f77712b8c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:39.391405 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f042d149-86a2-45d2-a75a-301f77712b8c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:39.392232 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f042d149-86a2-45d2-a75a-301f77712b8c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:39.392745 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f042d149-86a2-45d2-a75a-301f77712b8c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:39.525143 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-64186307-4574-4c96-ad0d-73ffa7c91f22 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:39.533849 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-64186307-4574-4c96-ad0d-73ffa7c91f22 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.186494 Jul 11 14:43:39.534036 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f042d149-86a2-45d2-a75a-301f77712b8c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:39.534157 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 228/477] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:39 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:39.555005 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f042d149-86a2-45d2-a75a-301f77712b8c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1318 microversion: 2.1 time: 0.178767 Jul 11 14:43:39.556165 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 250/478] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:39 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1318 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:40.027370 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7e397bdd-12ee-4c4a-95b8-0e697e69aad9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:40.028303 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-7e397bdd-12ee-4c4a-95b8-0e697e69aad9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:40.034624 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7e397bdd-12ee-4c4a-95b8-0e697e69aad9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:40.035080 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7e397bdd-12ee-4c4a-95b8-0e697e69aad9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:40.035854 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7e397bdd-12ee-4c4a-95b8-0e697e69aad9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:40.036294 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7e397bdd-12ee-4c4a-95b8-0e697e69aad9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:40.174318 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7e397bdd-12ee-4c4a-95b8-0e697e69aad9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:40.184302 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7e397bdd-12ee-4c4a-95b8-0e697e69aad9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:40.186180 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7e397bdd-12ee-4c4a-95b8-0e697e69aad9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:40.187464 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-7e397bdd-12ee-4c4a-95b8-0e697e69aad9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.164042 Jul 11 14:43:40.188104 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 229/479] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:40 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:40.553118 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2a8ec279-9045-49bd-a42b-8d767a5de195 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:40.553779 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2a8ec279-9045-49bd-a42b-8d767a5de195 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:40.562092 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2a8ec279-9045-49bd-a42b-8d767a5de195 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:40.562701 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2a8ec279-9045-49bd-a42b-8d767a5de195 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:40.563602 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2a8ec279-9045-49bd-a42b-8d767a5de195 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:40.564169 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2a8ec279-9045-49bd-a42b-8d767a5de195 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:40.576491 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e1929455-7bb0-4040-8b9a-939134e11363 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:40.577843 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-e1929455-7bb0-4040-8b9a-939134e11363 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:40.584989 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=87,nova_api:INSERT=3 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:40.586807 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e1929455-7bb0-4040-8b9a-939134e11363 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:40.587302 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e1929455-7bb0-4040-8b9a-939134e11363 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:40.588106 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e1929455-7bb0-4040-8b9a-939134e11363 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:40.588545 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e1929455-7bb0-4040-8b9a-939134e11363 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:40.714012 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-2a8ec279-9045-49bd-a42b-8d767a5de195 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:40.722023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2a8ec279-9045-49bd-a42b-8d767a5de195 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.173712 Jul 11 14:43:40.723228 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 251/480] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:40 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:40.749532 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e1929455-7bb0-4040-8b9a-939134e11363 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:40.757189 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e1929455-7bb0-4040-8b9a-939134e11363 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1318 microversion: 2.1 time: 0.189177 Jul 11 14:43:40.757979 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 230/481] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:40 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1318 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:41.205706 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-bdde2c04-ca67-4e71-abb7-f305bda0bf9c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:41.206582 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-bdde2c04-ca67-4e71-abb7-f305bda0bf9c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:41.213475 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bdde2c04-ca67-4e71-abb7-f305bda0bf9c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:41.214008 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bdde2c04-ca67-4e71-abb7-f305bda0bf9c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:41.214830 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bdde2c04-ca67-4e71-abb7-f305bda0bf9c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:41.215262 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bdde2c04-ca67-4e71-abb7-f305bda0bf9c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:41.353780 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-bdde2c04-ca67-4e71-abb7-f305bda0bf9c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:41.362213 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-bdde2c04-ca67-4e71-abb7-f305bda0bf9c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:41.363778 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-bdde2c04-ca67-4e71-abb7-f305bda0bf9c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:41.364937 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-bdde2c04-ca67-4e71-abb7-f305bda0bf9c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.163302 Jul 11 14:43:41.365499 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 252/482] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:41 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:41.739366 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f6c06df1-98c9-45ad-9e38-34ebba03a076 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:41.740011 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f6c06df1-98c9-45ad-9e38-34ebba03a076 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:41.746801 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f6c06df1-98c9-45ad-9e38-34ebba03a076 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:41.747264 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f6c06df1-98c9-45ad-9e38-34ebba03a076 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:41.748042 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f6c06df1-98c9-45ad-9e38-34ebba03a076 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:41.748482 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f6c06df1-98c9-45ad-9e38-34ebba03a076 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:41.774581 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-fd046bb7-3b51-454f-9133-9612b62190b8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:41.775297 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-fd046bb7-3b51-454f-9133-9612b62190b8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:41.781854 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fd046bb7-3b51-454f-9133-9612b62190b8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:41.782304 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fd046bb7-3b51-454f-9133-9612b62190b8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:41.783103 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fd046bb7-3b51-454f-9133-9612b62190b8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:41.783538 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fd046bb7-3b51-454f-9133-9612b62190b8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:41.891886 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f6c06df1-98c9-45ad-9e38-34ebba03a076 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:41.901992 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f6c06df1-98c9-45ad-9e38-34ebba03a076 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.166335 Jul 11 14:43:41.903388 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 231/483] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:41 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:41.920913 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-fd046bb7-3b51-454f-9133-9612b62190b8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:41.928701 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-fd046bb7-3b51-454f-9133-9612b62190b8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1318 microversion: 2.1 time: 0.158479 Jul 11 14:43:41.929251 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 253/484] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:41 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1318 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:42.383952 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-bd33cc4a-d1ac-4961-9e06-7cf4715e6046 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:42.384562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-bd33cc4a-d1ac-4961-9e06-7cf4715e6046 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:42.390857 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bd33cc4a-d1ac-4961-9e06-7cf4715e6046 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:42.391306 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bd33cc4a-d1ac-4961-9e06-7cf4715e6046 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:42.392084 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bd33cc4a-d1ac-4961-9e06-7cf4715e6046 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:42.392521 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bd33cc4a-d1ac-4961-9e06-7cf4715e6046 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:42.564653 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-bd33cc4a-d1ac-4961-9e06-7cf4715e6046 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:42.574826 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-bd33cc4a-d1ac-4961-9e06-7cf4715e6046 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:42.576700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-bd33cc4a-d1ac-4961-9e06-7cf4715e6046 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:42.577896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-bd33cc4a-d1ac-4961-9e06-7cf4715e6046 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.197876 Jul 11 14:43:42.578979 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 232/485] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:42 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:42.920220 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-ae4679ea-a380-478d-9aa8-a843a365056d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:42.920813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-ae4679ea-a380-478d-9aa8-a843a365056d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:42.922587 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=2 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:42.923273 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=14 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:42.928891 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ae4679ea-a380-478d-9aa8-a843a365056d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:42.929346 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ae4679ea-a380-478d-9aa8-a843a365056d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:42.930623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ae4679ea-a380-478d-9aa8-a843a365056d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:42.931063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ae4679ea-a380-478d-9aa8-a843a365056d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:42.945862 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-b3185365-3a9d-4442-b23e-77acc815193e tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:42.947041 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-b3185365-3a9d-4442-b23e-77acc815193e tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:42.954123 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b3185365-3a9d-4442-b23e-77acc815193e tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:42.954619 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b3185365-3a9d-4442-b23e-77acc815193e tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:42.955702 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b3185365-3a9d-4442-b23e-77acc815193e tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:42.956497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b3185365-3a9d-4442-b23e-77acc815193e tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:43.085639 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-ae4679ea-a380-478d-9aa8-a843a365056d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:43.098751 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-ae4679ea-a380-478d-9aa8-a843a365056d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.182005 Jul 11 14:43:43.099522 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 254/486] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:42 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:43.105424 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-b3185365-3a9d-4442-b23e-77acc815193e tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:43.113446 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-b3185365-3a9d-4442-b23e-77acc815193e tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1318 microversion: 2.1 time: 0.171364 Jul 11 14:43:43.114150 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 233/487] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:42 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1318 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:43.595720 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-492d3f08-514e-4a6f-b57c-f4e9b61b69cc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:43.596337 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-492d3f08-514e-4a6f-b57c-f4e9b61b69cc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:43.603219 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-492d3f08-514e-4a6f-b57c-f4e9b61b69cc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:43.603773 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-492d3f08-514e-4a6f-b57c-f4e9b61b69cc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:43.604430 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-492d3f08-514e-4a6f-b57c-f4e9b61b69cc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:43.604866 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-492d3f08-514e-4a6f-b57c-f4e9b61b69cc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:43.746509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-492d3f08-514e-4a6f-b57c-f4e9b61b69cc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:43.755764 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-492d3f08-514e-4a6f-b57c-f4e9b61b69cc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:43.757286 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-492d3f08-514e-4a6f-b57c-f4e9b61b69cc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:43.758515 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-492d3f08-514e-4a6f-b57c-f4e9b61b69cc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.166571 Jul 11 14:43:43.759073 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 255/488] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:43 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:44.118559 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-4c721275-31d9-4590-92c3-1f7fdfdef90f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:44.119185 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-4c721275-31d9-4590-92c3-1f7fdfdef90f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:44.125633 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4c721275-31d9-4590-92c3-1f7fdfdef90f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:44.130897 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-49b99ae8-7147-48ad-b256-7a04d8cf2a09 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:44.131527 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-49b99ae8-7147-48ad-b256-7a04d8cf2a09 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:44.135952 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4c721275-31d9-4590-92c3-1f7fdfdef90f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:44.136819 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4c721275-31d9-4590-92c3-1f7fdfdef90f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:44.137260 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4c721275-31d9-4590-92c3-1f7fdfdef90f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:44.141894 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-49b99ae8-7147-48ad-b256-7a04d8cf2a09 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:44.142348 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-49b99ae8-7147-48ad-b256-7a04d8cf2a09 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:44.143163 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-49b99ae8-7147-48ad-b256-7a04d8cf2a09 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:44.143605 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-49b99ae8-7147-48ad-b256-7a04d8cf2a09 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:44.277841 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4c721275-31d9-4590-92c3-1f7fdfdef90f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:44.286875 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-49b99ae8-7147-48ad-b256-7a04d8cf2a09 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:44.295114 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-49b99ae8-7147-48ad-b256-7a04d8cf2a09 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1318 microversion: 2.1 time: 0.168010 Jul 11 14:43:44.298111 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-4c721275-31d9-4590-92c3-1f7fdfdef90f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.182277 Jul 11 14:43:44.298277 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 256/489] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:44 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1318 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:44.298388 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 234/490] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:44 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:44.776842 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3285a11b-4939-4303-b774-4d9fdf89e826 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:44.777572 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3285a11b-4939-4303-b774-4d9fdf89e826 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:44.783849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3285a11b-4939-4303-b774-4d9fdf89e826 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:44.784372 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3285a11b-4939-4303-b774-4d9fdf89e826 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:44.785231 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3285a11b-4939-4303-b774-4d9fdf89e826 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:44.785851 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3285a11b-4939-4303-b774-4d9fdf89e826 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:44.928830 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3285a11b-4939-4303-b774-4d9fdf89e826 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:44.938169 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3285a11b-4939-4303-b774-4d9fdf89e826 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:44.939734 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3285a11b-4939-4303-b774-4d9fdf89e826 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:44.940905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3285a11b-4939-4303-b774-4d9fdf89e826 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.168257 Jul 11 14:43:44.941464 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 257/491] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:44 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:45.313901 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e8a63b9d-3306-4c43-9acf-f54ff5a3767b tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:45.314618 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c11dc516-f568-47de-b0e4-0a65b8e85ed9 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:45.315820 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c11dc516-f568-47de-b0e4-0a65b8e85ed9 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:45.316359 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-e8a63b9d-3306-4c43-9acf-f54ff5a3767b tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:45.323029 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c11dc516-f568-47de-b0e4-0a65b8e85ed9 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:45.323316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e8a63b9d-3306-4c43-9acf-f54ff5a3767b tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:45.323487 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c11dc516-f568-47de-b0e4-0a65b8e85ed9 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:45.323773 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e8a63b9d-3306-4c43-9acf-f54ff5a3767b tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:45.324263 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c11dc516-f568-47de-b0e4-0a65b8e85ed9 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:45.324546 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e8a63b9d-3306-4c43-9acf-f54ff5a3767b tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:45.324709 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c11dc516-f568-47de-b0e4-0a65b8e85ed9 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:45.324988 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e8a63b9d-3306-4c43-9acf-f54ff5a3767b tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:45.341203 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=100 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:45.463522 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e8a63b9d-3306-4c43-9acf-f54ff5a3767b tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:45.468047 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c11dc516-f568-47de-b0e4-0a65b8e85ed9 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:45.471641 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e8a63b9d-3306-4c43-9acf-f54ff5a3767b tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1318 microversion: 2.1 time: 0.161649 Jul 11 14:43:45.472426 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 235/492] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:45 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1318 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:45.475712 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c11dc516-f568-47de-b0e4-0a65b8e85ed9 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.165386 Jul 11 14:43:45.476710 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 258/493] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:45 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:45.959731 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-76a7c073-695c-405f-b26b-22eb94021110 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:45.960262 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-76a7c073-695c-405f-b26b-22eb94021110 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:45.966897 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-76a7c073-695c-405f-b26b-22eb94021110 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:45.967350 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-76a7c073-695c-405f-b26b-22eb94021110 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:45.968118 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-76a7c073-695c-405f-b26b-22eb94021110 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:45.968559 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-76a7c073-695c-405f-b26b-22eb94021110 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:46.101631 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-76a7c073-695c-405f-b26b-22eb94021110 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:46.110926 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-76a7c073-695c-405f-b26b-22eb94021110 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:46.112473 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-76a7c073-695c-405f-b26b-22eb94021110 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:46.113647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-76a7c073-695c-405f-b26b-22eb94021110 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.158193 Jul 11 14:43:46.114227 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 236/494] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:45 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:46.493045 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d7c0f1b4-99d4-43de-a490-8f7b275f4551 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:46.493752 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d7c0f1b4-99d4-43de-a490-8f7b275f4551 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:46.498333 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1ccff59c-bbae-40d2-aed9-47be15daee50 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:46.499000 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1ccff59c-bbae-40d2-aed9-47be15daee50 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:46.505541 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d7c0f1b4-99d4-43de-a490-8f7b275f4551 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:46.505999 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d7c0f1b4-99d4-43de-a490-8f7b275f4551 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:46.510115 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1ccff59c-bbae-40d2-aed9-47be15daee50 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:46.511377 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1ccff59c-bbae-40d2-aed9-47be15daee50 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:46.512184 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1ccff59c-bbae-40d2-aed9-47be15daee50 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:46.512494 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d7c0f1b4-99d4-43de-a490-8f7b275f4551 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:46.512619 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1ccff59c-bbae-40d2-aed9-47be15daee50 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:46.513223 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d7c0f1b4-99d4-43de-a490-8f7b275f4551 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:46.750156 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1ccff59c-bbae-40d2-aed9-47be15daee50 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:46.759554 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1ccff59c-bbae-40d2-aed9-47be15daee50 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1318 microversion: 2.1 time: 0.274469 Jul 11 14:43:46.760472 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 259/495] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:46 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1318 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:46.793426 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d7c0f1b4-99d4-43de-a490-8f7b275f4551 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:46.801283 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d7c0f1b4-99d4-43de-a490-8f7b275f4551 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.312161 Jul 11 14:43:46.801950 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 237/496] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:46 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:47.027903 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-da564e3d-c864-4fff-9f8a-1c5ebd20e3c5 req-dcfed557-caf2-4836-85a8-2c135f9cbf12 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0033415c-3adf-4fe8-bac6-60cf7774143f", "tag": "eacb6063-c62f-4742-8ca6-7f93d25fdaf8"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:43:47.037602 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-da564e3d-c864-4fff-9f8a-1c5ebd20e3c5 req-dcfed557-caf2-4836-85a8-2c135f9cbf12 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:47.038047 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-da564e3d-c864-4fff-9f8a-1c5ebd20e3c5 req-dcfed557-caf2-4836-85a8-2c135f9cbf12 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:47.055321 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-da564e3d-c864-4fff-9f8a-1c5ebd20e3c5 req-dcfed557-caf2-4836-85a8-2c135f9cbf12 service nova] Creating event network-changed:eacb6063-c62f-4742-8ca6-7f93d25fdaf8 for instance 0033415c-3adf-4fe8-bac6-60cf7774143f on hv-cinder-maste Jul 11 14:43:47.061205 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-da564e3d-c864-4fff-9f8a-1c5ebd20e3c5 req-dcfed557-caf2-4836-85a8-2c135f9cbf12 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037455 Jul 11 14:43:47.062139 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 260/497] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:43:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:43:47.132131 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8375f1d0-e641-4d7a-b0ec-8e7cf841f804 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:47.132754 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-8375f1d0-e641-4d7a-b0ec-8e7cf841f804 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:47.139902 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8375f1d0-e641-4d7a-b0ec-8e7cf841f804 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:47.140359 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8375f1d0-e641-4d7a-b0ec-8e7cf841f804 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:47.141646 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8375f1d0-e641-4d7a-b0ec-8e7cf841f804 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:47.141646 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8375f1d0-e641-4d7a-b0ec-8e7cf841f804 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:47.147496 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-99216cf8-8f85-4df3-af1c-ed115b5b6ba5 req-399ee9fb-952f-42a4-aead-7ffee54279b0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7e398a3c-0f26-422e-8cb2-828013595327", "tag": "1e493ea0-ba58-44b8-906b-71b97862dd76"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:43:47.160340 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-99216cf8-8f85-4df3-af1c-ed115b5b6ba5 req-399ee9fb-952f-42a4-aead-7ffee54279b0 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:47.161168 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-99216cf8-8f85-4df3-af1c-ed115b5b6ba5 req-399ee9fb-952f-42a4-aead-7ffee54279b0 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:47.186020 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-99216cf8-8f85-4df3-af1c-ed115b5b6ba5 req-399ee9fb-952f-42a4-aead-7ffee54279b0 service nova] Creating event network-changed:1e493ea0-ba58-44b8-906b-71b97862dd76 for instance 7e398a3c-0f26-422e-8cb2-828013595327 on hv-cinder-maste Jul 11 14:43:47.191136 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-99216cf8-8f85-4df3-af1c-ed115b5b6ba5 req-399ee9fb-952f-42a4-aead-7ffee54279b0 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050287 Jul 11 14:43:47.192077 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 261/498] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:43:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:43:47.292126 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8375f1d0-e641-4d7a-b0ec-8e7cf841f804 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:47.300588 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8375f1d0-e641-4d7a-b0ec-8e7cf841f804 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:47.302101 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8375f1d0-e641-4d7a-b0ec-8e7cf841f804 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:47.303289 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8375f1d0-e641-4d7a-b0ec-8e7cf841f804 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.175148 Jul 11 14:43:47.303848 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 238/499] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:47 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:47.779159 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-efd856b6-ab5f-4dc4-af5e-21adaf9479f7 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:47.779893 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-efd856b6-ab5f-4dc4-af5e-21adaf9479f7 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:47.787448 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-efd856b6-ab5f-4dc4-af5e-21adaf9479f7 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:47.787897 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-efd856b6-ab5f-4dc4-af5e-21adaf9479f7 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:47.788797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-efd856b6-ab5f-4dc4-af5e-21adaf9479f7 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:47.789400 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-efd856b6-ab5f-4dc4-af5e-21adaf9479f7 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:47.819239 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-abcf06bf-d4c7-438c-bf5a-6fea5142f6cf tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:47.819866 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-abcf06bf-d4c7-438c-bf5a-6fea5142f6cf tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:47.830004 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-abcf06bf-d4c7-438c-bf5a-6fea5142f6cf tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:47.830768 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-abcf06bf-d4c7-438c-bf5a-6fea5142f6cf tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:47.832092 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-abcf06bf-d4c7-438c-bf5a-6fea5142f6cf tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:47.832686 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-abcf06bf-d4c7-438c-bf5a-6fea5142f6cf tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:48.098838 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-abcf06bf-d4c7-438c-bf5a-6fea5142f6cf tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:48.107641 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-abcf06bf-d4c7-438c-bf5a-6fea5142f6cf tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.292336 Jul 11 14:43:48.108570 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 239/500] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:47 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:48.269521 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-efd856b6-ab5f-4dc4-af5e-21adaf9479f7 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:48.277408 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-efd856b6-ab5f-4dc4-af5e-21adaf9479f7 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1318 microversion: 2.1 time: 0.503595 Jul 11 14:43:48.287121 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 262/501] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:47 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1318 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:48.321852 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-fedfb5fb-ce76-4d14-8e02-794ce6e9d54e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:48.322530 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-fedfb5fb-ce76-4d14-8e02-794ce6e9d54e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:48.328872 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fedfb5fb-ce76-4d14-8e02-794ce6e9d54e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:48.329327 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fedfb5fb-ce76-4d14-8e02-794ce6e9d54e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:48.330107 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fedfb5fb-ce76-4d14-8e02-794ce6e9d54e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:48.330571 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fedfb5fb-ce76-4d14-8e02-794ce6e9d54e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:48.467812 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-fedfb5fb-ce76-4d14-8e02-794ce6e9d54e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:48.476514 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-fedfb5fb-ce76-4d14-8e02-794ce6e9d54e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:48.478606 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-fedfb5fb-ce76-4d14-8e02-794ce6e9d54e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:48.479759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-fedfb5fb-ce76-4d14-8e02-794ce6e9d54e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.162405 Jul 11 14:43:48.480316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 240/502] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:48 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:49.124502 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a12cb064-87c2-4581-968b-9a65ab046b59 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:49.125131 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a12cb064-87c2-4581-968b-9a65ab046b59 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:49.131697 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a12cb064-87c2-4581-968b-9a65ab046b59 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:49.132150 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a12cb064-87c2-4581-968b-9a65ab046b59 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:49.132914 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a12cb064-87c2-4581-968b-9a65ab046b59 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:49.133350 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a12cb064-87c2-4581-968b-9a65ab046b59 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:49.263539 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a12cb064-87c2-4581-968b-9a65ab046b59 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:49.270909 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a12cb064-87c2-4581-968b-9a65ab046b59 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.150341 Jul 11 14:43:49.271474 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 263/503] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:49 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:49.294465 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7575faaa-8383-4d67-9f21-09c2082c629a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:49.295124 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-7575faaa-8383-4d67-9f21-09c2082c629a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:49.301504 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7575faaa-8383-4d67-9f21-09c2082c629a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:49.302000 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7575faaa-8383-4d67-9f21-09c2082c629a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:49.302766 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7575faaa-8383-4d67-9f21-09c2082c629a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:49.303205 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7575faaa-8383-4d67-9f21-09c2082c629a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:49.432968 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7575faaa-8383-4d67-9f21-09c2082c629a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:49.440539 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-7575faaa-8383-4d67-9f21-09c2082c629a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1318 microversion: 2.1 time: 0.149942 Jul 11 14:43:49.441244 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 241/504] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:49 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1318 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:49.497099 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-babfa66d-a6d0-49cf-9fda-13da0baf388d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:49.497705 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-babfa66d-a6d0-49cf-9fda-13da0baf388d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:49.503971 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-babfa66d-a6d0-49cf-9fda-13da0baf388d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:49.504430 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-babfa66d-a6d0-49cf-9fda-13da0baf388d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:49.505231 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-babfa66d-a6d0-49cf-9fda-13da0baf388d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:49.505635 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-babfa66d-a6d0-49cf-9fda-13da0baf388d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:49.635979 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-babfa66d-a6d0-49cf-9fda-13da0baf388d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:49.644552 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-babfa66d-a6d0-49cf-9fda-13da0baf388d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:49.646085 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-babfa66d-a6d0-49cf-9fda-13da0baf388d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:49.647298 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-babfa66d-a6d0-49cf-9fda-13da0baf388d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.154175 Jul 11 14:43:49.657648 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 264/505] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:49 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:50.289191 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-1ab8576e-27a0-40a5-9113-ec969259ec6e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:50.291660 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-1ab8576e-27a0-40a5-9113-ec969259ec6e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:50.295940 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1ab8576e-27a0-40a5-9113-ec969259ec6e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:50.296529 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1ab8576e-27a0-40a5-9113-ec969259ec6e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:50.297135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1ab8576e-27a0-40a5-9113-ec969259ec6e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:50.297566 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1ab8576e-27a0-40a5-9113-ec969259ec6e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:50.425205 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-1ab8576e-27a0-40a5-9113-ec969259ec6e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:50.433128 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-1ab8576e-27a0-40a5-9113-ec969259ec6e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.147995 Jul 11 14:43:50.433689 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 242/506] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:50 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:50.457287 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-35bbfa36-4f05-461b-84fe-a40227b39fe3 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:50.457897 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-35bbfa36-4f05-461b-84fe-a40227b39fe3 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:50.464529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-35bbfa36-4f05-461b-84fe-a40227b39fe3 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:50.464971 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-35bbfa36-4f05-461b-84fe-a40227b39fe3 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:50.465724 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-35bbfa36-4f05-461b-84fe-a40227b39fe3 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:50.466155 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-35bbfa36-4f05-461b-84fe-a40227b39fe3 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:50.598977 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-35bbfa36-4f05-461b-84fe-a40227b39fe3 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:50.606979 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-35bbfa36-4f05-461b-84fe-a40227b39fe3 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1318 microversion: 2.1 time: 0.153794 Jul 11 14:43:50.607557 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 265/507] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:50 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1318 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:50.665859 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-24679a4b-aa7a-4b64-b981-1d7ca637952b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:50.666271 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-24679a4b-aa7a-4b64-b981-1d7ca637952b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:50.675049 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-24679a4b-aa7a-4b64-b981-1d7ca637952b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:50.675505 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-24679a4b-aa7a-4b64-b981-1d7ca637952b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:50.676462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-24679a4b-aa7a-4b64-b981-1d7ca637952b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:50.677064 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-24679a4b-aa7a-4b64-b981-1d7ca637952b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:50.818990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-24679a4b-aa7a-4b64-b981-1d7ca637952b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:50.834982 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-24679a4b-aa7a-4b64-b981-1d7ca637952b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:50.836630 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-24679a4b-aa7a-4b64-b981-1d7ca637952b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:50.837848 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-24679a4b-aa7a-4b64-b981-1d7ca637952b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.176671 Jul 11 14:43:50.838768 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 243/508] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:50 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:50.845516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-1a85173d-348b-4fbd-b212-c25041abc989 req-552948a6-a1dd-4adb-b1b7-72bfe4fe0ee2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "13da005a-e085-43aa-8a18-1fe85e4e95af", "name": "network-vif-plugged", "status": "completed", "tag": "7b725b13-befd-4a14-98cc-9d0e599f5e17"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:43:50.862012 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-1a85173d-348b-4fbd-b212-c25041abc989 req-552948a6-a1dd-4adb-b1b7-72bfe4fe0ee2 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:50.862818 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-1a85173d-348b-4fbd-b212-c25041abc989 req-552948a6-a1dd-4adb-b1b7-72bfe4fe0ee2 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:50.876572 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-1a85173d-348b-4fbd-b212-c25041abc989 req-552948a6-a1dd-4adb-b1b7-72bfe4fe0ee2 service nova] Creating event network-vif-plugged:7b725b13-befd-4a14-98cc-9d0e599f5e17 for instance 13da005a-e085-43aa-8a18-1fe85e4e95af on hv-cinder-maste Jul 11 14:43:50.881600 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-1a85173d-348b-4fbd-b212-c25041abc989 req-552948a6-a1dd-4adb-b1b7-72bfe4fe0ee2 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044977 Jul 11 14:43:50.882199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 266/509] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:43:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:43:51.449574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-ea30f70a-e1b7-4465-ba5e-163fc7cc10d0 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:51.450191 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-ea30f70a-e1b7-4465-ba5e-163fc7cc10d0 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:51.457072 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ea30f70a-e1b7-4465-ba5e-163fc7cc10d0 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:51.457520 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ea30f70a-e1b7-4465-ba5e-163fc7cc10d0 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:51.458304 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ea30f70a-e1b7-4465-ba5e-163fc7cc10d0 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:51.458773 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ea30f70a-e1b7-4465-ba5e-163fc7cc10d0 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:51.526569 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9211e759-5199-41f0-b8f8-d3731150b1bd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:51.527103 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9211e759-5199-41f0-b8f8-d3731150b1bd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:51.533529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9211e759-5199-41f0-b8f8-d3731150b1bd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:51.533989 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9211e759-5199-41f0-b8f8-d3731150b1bd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:51.534799 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9211e759-5199-41f0-b8f8-d3731150b1bd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:51.535243 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9211e759-5199-41f0-b8f8-d3731150b1bd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:51.611508 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-ea30f70a-e1b7-4465-ba5e-163fc7cc10d0 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:51.619658 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-ea30f70a-e1b7-4465-ba5e-163fc7cc10d0 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.174199 Jul 11 14:43:51.620281 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 244/510] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:51 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:51.625017 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-6c808e94-429e-452c-a0ea-f672bd5a62e0 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:51.625639 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-6c808e94-429e-452c-a0ea-f672bd5a62e0 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:51.631697 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6c808e94-429e-452c-a0ea-f672bd5a62e0 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:51.632249 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6c808e94-429e-452c-a0ea-f672bd5a62e0 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:51.633425 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6c808e94-429e-452c-a0ea-f672bd5a62e0 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:51.633872 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6c808e94-429e-452c-a0ea-f672bd5a62e0 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:51.638674 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=100 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:51.643213 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.objects.instance [None req-9211e759-5199-41f0-b8f8-d3731150b1bd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lazy-loading 'services' on Instance uuid 62a90c58-0027-47e8-8909-44512060e08d {{(pid=130159) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:43:51.718638 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9211e759-5199-41f0-b8f8-d3731150b1bd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "DELETE /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d/os-volume_attachments/05e12cf2-ae62-436e-b69f-67ab88493cb5" status: 202 len: 0 microversion: 2.1 time: 0.197380 Jul 11 14:43:51.719480 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 267/511] 10.222.0.72 () {58 vars in 1415 bytes} [Mon Jul 11 14:43:51 2022] DELETE /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d/os-volume_attachments/05e12cf2-ae62-436e-b69f-67ab88493cb5 => generated 0 bytes in 199 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 11 14:43:51.772362 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6c808e94-429e-452c-a0ea-f672bd5a62e0 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:51.780614 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-6c808e94-429e-452c-a0ea-f672bd5a62e0 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1318 microversion: 2.1 time: 0.159206 Jul 11 14:43:51.781150 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 245/512] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:51 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1318 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:51.864533 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3455984a-cd0f-40d4-a085-99fafe259f08 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:51.865250 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3455984a-cd0f-40d4-a085-99fafe259f08 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:51.872195 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3455984a-cd0f-40d4-a085-99fafe259f08 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:51.872792 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3455984a-cd0f-40d4-a085-99fafe259f08 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:51.873559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3455984a-cd0f-40d4-a085-99fafe259f08 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:51.873997 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3455984a-cd0f-40d4-a085-99fafe259f08 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:52.019304 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3455984a-cd0f-40d4-a085-99fafe259f08 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:52.027739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3455984a-cd0f-40d4-a085-99fafe259f08 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:52.029280 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3455984a-cd0f-40d4-a085-99fafe259f08 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:52.030462 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3455984a-cd0f-40d4-a085-99fafe259f08 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.170232 Jul 11 14:43:52.031068 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 268/513] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:51 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:52.637116 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2221e1d2-5177-4dea-84a0-d560e557fa7a tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:52.637798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2221e1d2-5177-4dea-84a0-d560e557fa7a tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:52.645241 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2221e1d2-5177-4dea-84a0-d560e557fa7a tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:52.645696 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2221e1d2-5177-4dea-84a0-d560e557fa7a tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:52.646510 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2221e1d2-5177-4dea-84a0-d560e557fa7a tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:52.646964 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2221e1d2-5177-4dea-84a0-d560e557fa7a tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:52.794944 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2221e1d2-5177-4dea-84a0-d560e557fa7a tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:52.803377 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-cb99f0d8-8740-4310-abe6-5f2d83d0b415 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:52.804145 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-cb99f0d8-8740-4310-abe6-5f2d83d0b415 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:52.809214 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2221e1d2-5177-4dea-84a0-d560e557fa7a tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.176532 Jul 11 14:43:52.810134 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 246/514] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:52 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:52.812122 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cb99f0d8-8740-4310-abe6-5f2d83d0b415 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:52.812674 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cb99f0d8-8740-4310-abe6-5f2d83d0b415 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:52.813688 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cb99f0d8-8740-4310-abe6-5f2d83d0b415 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:52.814206 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cb99f0d8-8740-4310-abe6-5f2d83d0b415 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:52.919860 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-5555c187-0689-48dd-854b-148e7a98c1db req-ce949e3b-b438-45a0-9d36-46b204fb6e24 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7e398a3c-0f26-422e-8cb2-828013595327", "name": "network-vif-plugged", "status": "completed", "tag": "1e493ea0-ba58-44b8-906b-71b97862dd76"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:43:52.932574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-5555c187-0689-48dd-854b-148e7a98c1db req-ce949e3b-b438-45a0-9d36-46b204fb6e24 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:52.933114 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-5555c187-0689-48dd-854b-148e7a98c1db req-ce949e3b-b438-45a0-9d36-46b204fb6e24 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:52.947149 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-5555c187-0689-48dd-854b-148e7a98c1db req-ce949e3b-b438-45a0-9d36-46b204fb6e24 service nova] Creating event network-vif-plugged:1e493ea0-ba58-44b8-906b-71b97862dd76 for instance 7e398a3c-0f26-422e-8cb2-828013595327 on hv-cinder-maste Jul 11 14:43:52.953096 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-5555c187-0689-48dd-854b-148e7a98c1db req-ce949e3b-b438-45a0-9d36-46b204fb6e24 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039907 Jul 11 14:43:52.953891 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 247/515] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:43:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:43:52.965541 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-cb99f0d8-8740-4310-abe6-5f2d83d0b415 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:52.974856 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-cb99f0d8-8740-4310-abe6-5f2d83d0b415 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1318 microversion: 2.1 time: 0.178531 Jul 11 14:43:52.975916 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 269/516] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:52 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1318 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:53.050110 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9b61a834-c0c6-41db-9992-50bd8fc53d24 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:53.050603 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-9b61a834-c0c6-41db-9992-50bd8fc53d24 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:53.057777 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9b61a834-c0c6-41db-9992-50bd8fc53d24 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:53.058240 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9b61a834-c0c6-41db-9992-50bd8fc53d24 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:53.059047 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9b61a834-c0c6-41db-9992-50bd8fc53d24 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:53.059667 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9b61a834-c0c6-41db-9992-50bd8fc53d24 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:53.207929 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-9b61a834-c0c6-41db-9992-50bd8fc53d24 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:53.216783 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-9b61a834-c0c6-41db-9992-50bd8fc53d24 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:53.218347 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-9b61a834-c0c6-41db-9992-50bd8fc53d24 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:53.219672 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9b61a834-c0c6-41db-9992-50bd8fc53d24 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.174527 Jul 11 14:43:53.220248 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 248/517] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:53 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:53.827206 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-308eb2c8-0f37-42b0-9098-f060ea3aa9e5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:53.827895 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-308eb2c8-0f37-42b0-9098-f060ea3aa9e5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:53.834945 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-308eb2c8-0f37-42b0-9098-f060ea3aa9e5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:53.835394 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-308eb2c8-0f37-42b0-9098-f060ea3aa9e5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:53.836707 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-308eb2c8-0f37-42b0-9098-f060ea3aa9e5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:53.837166 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-308eb2c8-0f37-42b0-9098-f060ea3aa9e5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:53.917851 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-207ad8fe-91ff-4963-ab93-470a2f95cba8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:43:53.929900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-207ad8fe-91ff-4963-ab93-470a2f95cba8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:53.930648 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-207ad8fe-91ff-4963-ab93-470a2f95cba8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:53.995395 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-308eb2c8-0f37-42b0-9098-f060ea3aa9e5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:54.000174 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-207ad8fe-91ff-4963-ab93-470a2f95cba8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Going to try to stop instance {{(pid=130160) force_stop /opt/stack/nova/nova/compute/api.py:2707}} Jul 11 14:43:54.003716 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-308eb2c8-0f37-42b0-9098-f060ea3aa9e5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.180494 Jul 11 14:43:54.004274 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 270/518] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:53 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:54.009535 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a9a8ad23-f4c1-41da-aabc-84a6476f62d8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:54.010452 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a9a8ad23-f4c1-41da-aabc-84a6476f62d8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:54.017066 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a9a8ad23-f4c1-41da-aabc-84a6476f62d8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:54.017603 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a9a8ad23-f4c1-41da-aabc-84a6476f62d8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:54.018450 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a9a8ad23-f4c1-41da-aabc-84a6476f62d8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:54.019101 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a9a8ad23-f4c1-41da-aabc-84a6476f62d8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:54.049647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-207ad8fe-91ff-4963-ab93-470a2f95cba8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "POST /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d/action" status: 202 len: 0 microversion: 2.1 time: 0.137003 Jul 11 14:43:54.053549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 249/519] 10.222.0.72 () {60 vars in 1277 bytes} [Mon Jul 11 14:43:53 2022] POST /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d/action => generated 0 bytes in 139 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:43:54.063181 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8e74cc59-6714-4e01-87b5-2f2f7b7b4b50 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:54.064177 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-8e74cc59-6714-4e01-87b5-2f2f7b7b4b50 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:54.071307 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8e74cc59-6714-4e01-87b5-2f2f7b7b4b50 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:54.071765 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8e74cc59-6714-4e01-87b5-2f2f7b7b4b50 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:54.072555 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8e74cc59-6714-4e01-87b5-2f2f7b7b4b50 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:54.072999 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8e74cc59-6714-4e01-87b5-2f2f7b7b4b50 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:54.172609 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a9a8ad23-f4c1-41da-aabc-84a6476f62d8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:54.180570 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a9a8ad23-f4c1-41da-aabc-84a6476f62d8 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1318 microversion: 2.1 time: 0.174552 Jul 11 14:43:54.181157 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 271/520] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:54 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1318 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:54.208296 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8e74cc59-6714-4e01-87b5-2f2f7b7b4b50 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:54.217773 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8e74cc59-6714-4e01-87b5-2f2f7b7b4b50 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1676 microversion: 2.1 time: 0.161674 Jul 11 14:43:54.218350 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 250/521] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:54 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1676 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:54.238081 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-dcb29d5b-6b67-4abe-aa40-484005fa00ff tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:54.238725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-dcb29d5b-6b67-4abe-aa40-484005fa00ff tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:54.245220 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-dcb29d5b-6b67-4abe-aa40-484005fa00ff tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:54.245690 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-dcb29d5b-6b67-4abe-aa40-484005fa00ff tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:54.246481 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-dcb29d5b-6b67-4abe-aa40-484005fa00ff tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:54.246896 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-dcb29d5b-6b67-4abe-aa40-484005fa00ff tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:54.387875 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-dcb29d5b-6b67-4abe-aa40-484005fa00ff tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:54.396647 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-dcb29d5b-6b67-4abe-aa40-484005fa00ff tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:54.398253 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-dcb29d5b-6b67-4abe-aa40-484005fa00ff tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:54.400069 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-dcb29d5b-6b67-4abe-aa40-484005fa00ff tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1808 microversion: 2.20 time: 0.165773 Jul 11 14:43:54.400701 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 272/522] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:54 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1808 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:43:54.968223 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-9f13f8c4-ec65-427d-aac7-1138a3023929 req-852c1a8b-0753-441e-af5f-300d4a2778a6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0033415c-3adf-4fe8-bac6-60cf7774143f", "name": "network-vif-plugged", "status": "completed", "tag": "eacb6063-c62f-4742-8ca6-7f93d25fdaf8"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:43:54.976860 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-9f13f8c4-ec65-427d-aac7-1138a3023929 req-852c1a8b-0753-441e-af5f-300d4a2778a6 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:54.977310 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-9f13f8c4-ec65-427d-aac7-1138a3023929 req-852c1a8b-0753-441e-af5f-300d4a2778a6 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:54.988773 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-9f13f8c4-ec65-427d-aac7-1138a3023929 req-852c1a8b-0753-441e-af5f-300d4a2778a6 service nova] Creating event network-vif-plugged:eacb6063-c62f-4742-8ca6-7f93d25fdaf8 for instance 0033415c-3adf-4fe8-bac6-60cf7774143f on hv-cinder-maste Jul 11 14:43:54.994810 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-9f13f8c4-ec65-427d-aac7-1138a3023929 req-852c1a8b-0753-441e-af5f-300d4a2778a6 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030110 Jul 11 14:43:54.995453 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 251/523] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:43:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:43:55.021120 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-b4f68797-5adc-419d-8d3d-d28f9d97d3a7 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:55.021775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-b4f68797-5adc-419d-8d3d-d28f9d97d3a7 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:55.028369 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b4f68797-5adc-419d-8d3d-d28f9d97d3a7 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:55.028817 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b4f68797-5adc-419d-8d3d-d28f9d97d3a7 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:55.029575 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b4f68797-5adc-419d-8d3d-d28f9d97d3a7 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:55.030042 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b4f68797-5adc-419d-8d3d-d28f9d97d3a7 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:55.164703 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-b4f68797-5adc-419d-8d3d-d28f9d97d3a7 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:55.172695 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-b4f68797-5adc-419d-8d3d-d28f9d97d3a7 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.155275 Jul 11 14:43:55.173562 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 273/524] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:55 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:55.198308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-028ff2e8-0593-459e-bc4a-54b827cdba8c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:55.199067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-028ff2e8-0593-459e-bc4a-54b827cdba8c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:55.205869 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-028ff2e8-0593-459e-bc4a-54b827cdba8c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:55.206613 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-028ff2e8-0593-459e-bc4a-54b827cdba8c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:55.207470 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-028ff2e8-0593-459e-bc4a-54b827cdba8c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:55.208184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-028ff2e8-0593-459e-bc4a-54b827cdba8c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:55.223553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:43:55.234323 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-923c4410-f0df-4b3f-a975-96e3851e6f75 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:55.234970 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-923c4410-f0df-4b3f-a975-96e3851e6f75 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:55.241334 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-923c4410-f0df-4b3f-a975-96e3851e6f75 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:55.241784 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-923c4410-f0df-4b3f-a975-96e3851e6f75 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:55.243217 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-923c4410-f0df-4b3f-a975-96e3851e6f75 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:55.243651 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-923c4410-f0df-4b3f-a975-96e3851e6f75 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:55.345080 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-028ff2e8-0593-459e-bc4a-54b827cdba8c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:55.353136 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-028ff2e8-0593-459e-bc4a-54b827cdba8c tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1318 microversion: 2.1 time: 0.159283 Jul 11 14:43:55.353715 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 252/525] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:55 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1318 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:55.383234 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-923c4410-f0df-4b3f-a975-96e3851e6f75 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:55.391469 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-923c4410-f0df-4b3f-a975-96e3851e6f75 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1676 microversion: 2.1 time: 0.160869 Jul 11 14:43:55.392276 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 274/526] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:55 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1676 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:55.419679 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-1bb597c4-7e3e-4f5c-aaba-c08610b8c604 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:55.420281 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-1bb597c4-7e3e-4f5c-aaba-c08610b8c604 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:55.426404 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1bb597c4-7e3e-4f5c-aaba-c08610b8c604 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:55.426912 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1bb597c4-7e3e-4f5c-aaba-c08610b8c604 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:55.427672 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1bb597c4-7e3e-4f5c-aaba-c08610b8c604 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:55.428103 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1bb597c4-7e3e-4f5c-aaba-c08610b8c604 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:55.562278 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-1bb597c4-7e3e-4f5c-aaba-c08610b8c604 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:55.570818 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-1bb597c4-7e3e-4f5c-aaba-c08610b8c604 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:55.572338 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-1bb597c4-7e3e-4f5c-aaba-c08610b8c604 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:55.573509 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-1bb597c4-7e3e-4f5c-aaba-c08610b8c604 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1795 microversion: 2.20 time: 0.158077 Jul 11 14:43:55.574082 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 253/527] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:43:55 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1795 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:43:56.190272 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-0e665a27-95d3-48f1-836b-dee0fd246f6f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:56.190899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-0e665a27-95d3-48f1-836b-dee0fd246f6f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:56.198053 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0e665a27-95d3-48f1-836b-dee0fd246f6f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:56.198537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0e665a27-95d3-48f1-836b-dee0fd246f6f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:56.199300 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0e665a27-95d3-48f1-836b-dee0fd246f6f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:56.199736 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0e665a27-95d3-48f1-836b-dee0fd246f6f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:56.340027 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0e665a27-95d3-48f1-836b-dee0fd246f6f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:56.348239 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-0e665a27-95d3-48f1-836b-dee0fd246f6f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.162152 Jul 11 14:43:56.348803 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 275/528] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:56 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:56.349395 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-054b42ad-5814-4c51-8879-5994261f5a9d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:56.349933 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-054b42ad-5814-4c51-8879-5994261f5a9d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:56.356728 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-054b42ad-5814-4c51-8879-5994261f5a9d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:56.357173 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-054b42ad-5814-4c51-8879-5994261f5a9d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:56.357938 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-054b42ad-5814-4c51-8879-5994261f5a9d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:56.358390 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-054b42ad-5814-4c51-8879-5994261f5a9d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:56.369547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c210700a-b238-4fac-bfba-48865824950d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:56.370181 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c210700a-b238-4fac-bfba-48865824950d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:56.376644 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c210700a-b238-4fac-bfba-48865824950d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:56.377104 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c210700a-b238-4fac-bfba-48865824950d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:56.377883 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c210700a-b238-4fac-bfba-48865824950d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:56.378320 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c210700a-b238-4fac-bfba-48865824950d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:56.384506 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-054b42ad-5814-4c51-8879-5994261f5a9d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562/ips" status: 200 len: 149 microversion: 2.1 time: 0.038737 Jul 11 14:43:56.385106 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 254/529] 10.222.0.72 () {58 vars in 1247 bytes} [Mon Jul 11 14:43:56 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562/ips => generated 149 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:43:56.395792 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-eb041072-31bd-4d05-abc1-478251c88d3d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "69ca4bc6-3d1c-4bef-ae85-7e63c008f48d"}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:43:56.396925 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-eb041072-31bd-4d05-abc1-478251c88d3d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:56.403451 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eb041072-31bd-4d05-abc1-478251c88d3d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:56.403906 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eb041072-31bd-4d05-abc1-478251c88d3d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:56.405030 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eb041072-31bd-4d05-abc1-478251c88d3d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:56.405204 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eb041072-31bd-4d05-abc1-478251c88d3d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:56.496828 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-eb041072-31bd-4d05-abc1-478251c88d3d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] HTTP exception thrown: Invalid volume: volume 69ca4bc6-3d1c-4bef-ae85-7e63c008f48d is already attached to instances: bd855cba-6708-47de-826b-00c5a74b23f2 Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-eb041072-31bd-4d05-abc1-478251c88d3d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Request method failure captured: Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: POST /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562/os-volume_attachments HTTP/1.1 Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 74 Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDZbQUJdl7M18phk4FM9-GGzi8XbGx24YhEK9KsOfI165n5v27J9wq7wzJjs1m7qkLCobj_5JRaSFQlNcVRL5jfwSqdoh1nodKIJxEOpM9gSJP97YpvVDnXLPsMNjwZQwG04QTf0uG5zLXZ7eKV_ivBXCJkCsdmhsPnE9ln-9M7wHs4 Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:43:56.499242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:43:56.500896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}] Jul 11 14:43:56.500896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:43:56.500896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:43:56.500896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:43:56.500896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:43:56.500896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:43:56.500896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:43:56.500896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 3ae56fd1172f4bf8806eefc6e8877fef Jul 11 14:43:56.500896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:43:56.500896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Jul 11 14:43:56.500896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {"volumeAttachment": {"volumeId": "69ca4bc6-3d1c-4bef-ae85-7e63c008f48d"}} Jul 11 14:43:56.500896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Invalid volume: volume 69ca4bc6-3d1c-4bef-ae85-7e63c008f48d is already attached to instances: bd855cba-6708-47de-826b-00c5a74b23f2 Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'server_id': '7a49d1ca-7491-41e0-b0d8-37c92b9d3562', 'body': {'volumeAttachment': {'volumeId': '69ca4bc6-3d1c-4bef-ae85-7e63c008f48d'}}} Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 69ca4bc6-3d1c-4bef-ae85-7e63c008f48d is already attached to instances: bd855cba-6708-47de-826b-00c5a74b23f2 Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:43:56.501811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 69ca4bc6-3d1c-4bef-ae85-7e63c008f48d is already attached to instances: bd855cba-6708-47de-826b-00c5a74b23f2 Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:43:56.503353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:43:56.504741 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:43:56.504741 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:43:56.504741 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:43:56.504741 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 69ca4bc6-3d1c-4bef-ae85-7e63c008f48d is already attached to instances: bd855cba-6708-47de-826b-00c5a74b23f2 Jul 11 14:43:56.504741 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:43:56.504741 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-eb041072-31bd-4d05-abc1-478251c88d3d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Returning 400 to user: Invalid volume: volume 69ca4bc6-3d1c-4bef-ae85-7e63c008f48d is already attached to instances: bd855cba-6708-47de-826b-00c5a74b23f2 {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:43:56.504741 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-eb041072-31bd-4d05-abc1-478251c88d3d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "POST /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.109198 Jul 11 14:43:56.505334 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 255/530] 10.222.0.72 () {60 vars in 1322 bytes} [Mon Jul 11 14:43:56 2022] POST /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562/os-volume_attachments => generated 174 bytes in 114 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Jul 11 14:43:56.510593 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-fbce5669-967f-4054-b100-b19f500ef699 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:56.511219 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-fbce5669-967f-4054-b100-b19f500ef699 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:56.518246 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fbce5669-967f-4054-b100-b19f500ef699 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:56.518676 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fbce5669-967f-4054-b100-b19f500ef699 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:56.519451 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fbce5669-967f-4054-b100-b19f500ef699 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:56.519894 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fbce5669-967f-4054-b100-b19f500ef699 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:56.527515 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c210700a-b238-4fac-bfba-48865824950d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:56.535696 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c210700a-b238-4fac-bfba-48865824950d tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1507 microversion: 2.1 time: 0.170464 Jul 11 14:43:56.536441 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 276/531] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:56 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1507 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:56.541399 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-5a857c76-fc6b-4312-bfe8-0569d93c6ac8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:56.542080 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-5a857c76-fc6b-4312-bfe8-0569d93c6ac8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:56.548754 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5a857c76-fc6b-4312-bfe8-0569d93c6ac8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:56.549330 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5a857c76-fc6b-4312-bfe8-0569d93c6ac8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:56.550202 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5a857c76-fc6b-4312-bfe8-0569d93c6ac8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:56.550737 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5a857c76-fc6b-4312-bfe8-0569d93c6ac8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:56.578210 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-5a857c76-fc6b-4312-bfe8-0569d93c6ac8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Going to try to terminate instance {{(pid=130159) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:43:56.655195 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-fbce5669-967f-4054-b100-b19f500ef699 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:56.658124 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-5a857c76-fc6b-4312-bfe8-0569d93c6ac8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "DELETE /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 204 len: 0 microversion: 2.1 time: 0.120367 Jul 11 14:43:56.658836 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 277/532] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:43:56 2022] DELETE /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 11 14:43:56.663790 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-fbce5669-967f-4054-b100-b19f500ef699 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1676 microversion: 2.1 time: 0.157531 Jul 11 14:43:56.664958 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 256/533] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:56 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1676 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:56.665290 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e727775a-c93c-4e90-9895-64619749476a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "593b97e2-90cf-4784-a426-92970e597a12", "device": "/dev/sdb"}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:43:56.666746 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e727775a-c93c-4e90-9895-64619749476a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:56.675779 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8cf6d436-ec31-4733-bfff-7d95703c1692 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:56.676205 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e727775a-c93c-4e90-9895-64619749476a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:56.676401 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-8cf6d436-ec31-4733-bfff-7d95703c1692 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:56.676792 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e727775a-c93c-4e90-9895-64619749476a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:56.677635 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e727775a-c93c-4e90-9895-64619749476a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:56.678205 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e727775a-c93c-4e90-9895-64619749476a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:56.683918 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8cf6d436-ec31-4733-bfff-7d95703c1692 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:56.684379 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8cf6d436-ec31-4733-bfff-7d95703c1692 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:56.685146 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8cf6d436-ec31-4733-bfff-7d95703c1692 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:56.685580 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8cf6d436-ec31-4733-bfff-7d95703c1692 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:56.822532 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8cf6d436-ec31-4733-bfff-7d95703c1692 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:56.830089 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8cf6d436-ec31-4733-bfff-7d95703c1692 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1680 microversion: 2.1 time: 0.163728 Jul 11 14:43:56.830703 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 257/534] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:56 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1680 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:57.365423 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9856025b-5e15-499b-aba9-148b1fc862d2 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:57.366067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-9856025b-5e15-499b-aba9-148b1fc862d2 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:57.373016 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9856025b-5e15-499b-aba9-148b1fc862d2 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:57.373471 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9856025b-5e15-499b-aba9-148b1fc862d2 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:57.374241 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9856025b-5e15-499b-aba9-148b1fc862d2 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:57.374718 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9856025b-5e15-499b-aba9-148b1fc862d2 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:57.507106 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-9856025b-5e15-499b-aba9-148b1fc862d2 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:57.515422 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9856025b-5e15-499b-aba9-148b1fc862d2 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.153904 Jul 11 14:43:57.516020 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 258/535] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:57 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:57.681609 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-5b3b461b-6515-4377-abc3-0f26e0040a71 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:57.682214 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-5b3b461b-6515-4377-abc3-0f26e0040a71 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:57.688209 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5b3b461b-6515-4377-abc3-0f26e0040a71 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:57.688652 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5b3b461b-6515-4377-abc3-0f26e0040a71 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:57.689398 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5b3b461b-6515-4377-abc3-0f26e0040a71 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:57.689829 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5b3b461b-6515-4377-abc3-0f26e0040a71 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:57.816054 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5b3b461b-6515-4377-abc3-0f26e0040a71 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:57.823556 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-5b3b461b-6515-4377-abc3-0f26e0040a71 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1676 microversion: 2.1 time: 0.146370 Jul 11 14:43:57.824118 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 259/536] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:57 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1676 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:57.847500 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f7af7304-95c4-447a-8320-81d87be702d1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:57.848105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f7af7304-95c4-447a-8320-81d87be702d1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:57.854638 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f7af7304-95c4-447a-8320-81d87be702d1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:57.855088 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f7af7304-95c4-447a-8320-81d87be702d1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:57.855851 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f7af7304-95c4-447a-8320-81d87be702d1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:57.856281 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f7af7304-95c4-447a-8320-81d87be702d1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:57.986515 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f7af7304-95c4-447a-8320-81d87be702d1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:57.994438 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f7af7304-95c4-447a-8320-81d87be702d1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1680 microversion: 2.1 time: 0.150563 Jul 11 14:43:57.995014 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 260/537] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:57 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1680 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:58.532406 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-4bac6b94-b27a-48c6-9232-b537986563a1 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:58.533035 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-4bac6b94-b27a-48c6-9232-b537986563a1 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:58.540043 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4bac6b94-b27a-48c6-9232-b537986563a1 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:58.540425 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4bac6b94-b27a-48c6-9232-b537986563a1 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:58.541174 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4bac6b94-b27a-48c6-9232-b537986563a1 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:58.541609 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4bac6b94-b27a-48c6-9232-b537986563a1 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:58.671113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4bac6b94-b27a-48c6-9232-b537986563a1 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:58.678892 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-4bac6b94-b27a-48c6-9232-b537986563a1 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1357 microversion: 2.1 time: 0.150333 Jul 11 14:43:58.679384 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 261/538] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:58 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1357 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:43:58.841371 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e8bc36e1-3a29-41e7-b1ac-8a18f0de0206 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:58.842127 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-e8bc36e1-3a29-41e7-b1ac-8a18f0de0206 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:58.848190 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e8bc36e1-3a29-41e7-b1ac-8a18f0de0206 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:58.848643 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e8bc36e1-3a29-41e7-b1ac-8a18f0de0206 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:58.849539 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e8bc36e1-3a29-41e7-b1ac-8a18f0de0206 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:58.850019 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e8bc36e1-3a29-41e7-b1ac-8a18f0de0206 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:58.866873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e727775a-c93c-4e90-9895-64619749476a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "POST /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.206467 Jul 11 14:43:58.868586 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 278/539] 10.222.0.72 () {60 vars in 1322 bytes} [Mon Jul 11 14:43:56 2022] POST /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327/os-volume_attachments => generated 194 bytes in 2209 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:43:58.991253 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e8bc36e1-3a29-41e7-b1ac-8a18f0de0206 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:59.000441 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e8bc36e1-3a29-41e7-b1ac-8a18f0de0206 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1676 microversion: 2.1 time: 0.163325 Jul 11 14:43:59.001024 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 262/540] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:58 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1676 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:59.012355 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1d1cca0e-23ef-4fe7-a109-ce728e9f3b1d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:59.013070 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1d1cca0e-23ef-4fe7-a109-ce728e9f3b1d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:59.019559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1d1cca0e-23ef-4fe7-a109-ce728e9f3b1d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:59.020023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1d1cca0e-23ef-4fe7-a109-ce728e9f3b1d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:59.020803 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1d1cca0e-23ef-4fe7-a109-ce728e9f3b1d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:59.021850 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1d1cca0e-23ef-4fe7-a109-ce728e9f3b1d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:59.157528 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1d1cca0e-23ef-4fe7-a109-ce728e9f3b1d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:59.166195 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1d1cca0e-23ef-4fe7-a109-ce728e9f3b1d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1680 microversion: 2.1 time: 0.158543 Jul 11 14:43:59.166836 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 279/541] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:59 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1680 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:59.695842 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-47abbaa2-d028-4b0a-9f8a-2f856126a012 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:59.696814 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-47abbaa2-d028-4b0a-9f8a-2f856126a012 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:59.704692 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-47abbaa2-d028-4b0a-9f8a-2f856126a012 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:59.705138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-47abbaa2-d028-4b0a-9f8a-2f856126a012 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:59.705906 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-47abbaa2-d028-4b0a-9f8a-2f856126a012 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:59.706641 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-47abbaa2-d028-4b0a-9f8a-2f856126a012 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:59.847669 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-47abbaa2-d028-4b0a-9f8a-2f856126a012 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:43:59.855629 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-47abbaa2-d028-4b0a-9f8a-2f856126a012 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1546 microversion: 2.1 time: 0.164204 Jul 11 14:43:59.856220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 263/542] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:59 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1546 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:43:59.871700 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-03cc488d-37e0-4215-9e19-e585a3efceca tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:43:59.872310 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-03cc488d-37e0-4215-9e19-e585a3efceca tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:43:59.878820 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-03cc488d-37e0-4215-9e19-e585a3efceca tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:59.879281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-03cc488d-37e0-4215-9e19-e585a3efceca tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:43:59.880049 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-03cc488d-37e0-4215-9e19-e585a3efceca tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:43:59.880488 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-03cc488d-37e0-4215-9e19-e585a3efceca tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:00.017485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-38b1665b-2d19-4957-b77b-3ac2c3facf0d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:00.018099 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-38b1665b-2d19-4957-b77b-3ac2c3facf0d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:00.024687 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-38b1665b-2d19-4957-b77b-3ac2c3facf0d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:00.025117 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-38b1665b-2d19-4957-b77b-3ac2c3facf0d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:00.025867 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-38b1665b-2d19-4957-b77b-3ac2c3facf0d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:00.026305 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-38b1665b-2d19-4957-b77b-3ac2c3facf0d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:00.143375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-03cc488d-37e0-4215-9e19-e585a3efceca tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:00.147719 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=3,nova_cell1:UPDATE=3 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:44:00.153238 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-03cc488d-37e0-4215-9e19-e585a3efceca tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1546 microversion: 2.1 time: 0.285291 Jul 11 14:44:00.154060 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 280/543] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:43:59 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1546 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:00.173787 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-38b1665b-2d19-4957-b77b-3ac2c3facf0d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:00.178860 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a4d06f10-6d31-4e07-b0a4-1a5ad84f8a77 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:00.183190 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a4d06f10-6d31-4e07-b0a4-1a5ad84f8a77 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:00.183938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a4d06f10-6d31-4e07-b0a4-1a5ad84f8a77 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Searching by: {'deleted': False, 'project_id': '3e77cfd4544c45468c57510de6034e38'} {{(pid=130159) get_all /opt/stack/nova/nova/compute/api.py:2962}} Jul 11 14:44:00.192374 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a4d06f10-6d31-4e07-b0a4-1a5ad84f8a77 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:00.193117 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a4d06f10-6d31-4e07-b0a4-1a5ad84f8a77 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:00.193420 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-38b1665b-2d19-4957-b77b-3ac2c3facf0d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1676 microversion: 2.1 time: 0.179876 Jul 11 14:44:00.194458 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 264/544] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:00 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1676 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:44:00.200949 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-da8aa48c-26f2-4c96-be14-ba8d6f442541 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:00.202703 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-da8aa48c-26f2-4c96-be14-ba8d6f442541 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:00.207399 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a4d06f10-6d31-4e07-b0a4-1a5ad84f8a77 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:00.208323 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a4d06f10-6d31-4e07-b0a4-1a5ad84f8a77 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:00.211168 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-da8aa48c-26f2-4c96-be14-ba8d6f442541 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:00.211915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-da8aa48c-26f2-4c96-be14-ba8d6f442541 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:00.213344 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-da8aa48c-26f2-4c96-be14-ba8d6f442541 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:00.213971 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-da8aa48c-26f2-4c96-be14-ba8d6f442541 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:00.311248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.multi_cell_list [None req-a4d06f10-6d31-4e07-b0a4-1a5ad84f8a77 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=130159) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Jul 11 14:44:00.362573 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-da8aa48c-26f2-4c96-be14-ba8d6f442541 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:00.372026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-da8aa48c-26f2-4c96-be14-ba8d6f442541 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1680 microversion: 2.1 time: 0.174990 Jul 11 14:44:00.378505 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 265/545] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:00 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1680 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:00.415770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a4d06f10-6d31-4e07-b0a4-1a5ad84f8a77 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers" status: 200 len: 342 microversion: 2.1 time: 0.249747 Jul 11 14:44:00.416372 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 281/546] 10.222.0.72 () {58 vars in 1124 bytes} [Mon Jul 11 14:44:00 2022] GET /compute/v2.1/servers => generated 342 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:44:00.426698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-0bb9830c-fe8b-490a-a185-9c3b15df1f08 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:00.427313 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-0bb9830c-fe8b-490a-a185-9c3b15df1f08 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:00.435077 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0bb9830c-fe8b-490a-a185-9c3b15df1f08 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:00.435530 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0bb9830c-fe8b-490a-a185-9c3b15df1f08 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:00.437452 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0bb9830c-fe8b-490a-a185-9c3b15df1f08 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:00.438060 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0bb9830c-fe8b-490a-a185-9c3b15df1f08 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:00.582300 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-0bb9830c-fe8b-490a-a185-9c3b15df1f08 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:00.590764 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-0bb9830c-fe8b-490a-a185-9c3b15df1f08 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1546 microversion: 2.1 time: 0.167924 Jul 11 14:44:00.594209 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 266/547] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:00 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1546 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:00.733418 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-1dd11762-de16-4dbd-a4d4-1c5c3e9f0f58 req-f28fc62a-9908-4e43-bfbd-a2c7af8632ad service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7a49d1ca-7491-41e0-b0d8-37c92b9d3562", "name": "network-vif-deleted", "tag": "c62c0fe9-fd78-4a72-aef8-9bd42e6d81de"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:44:00.755535 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-1dd11762-de16-4dbd-a4d4-1c5c3e9f0f58 req-f28fc62a-9908-4e43-bfbd-a2c7af8632ad service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:00.759168 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-1dd11762-de16-4dbd-a4d4-1c5c3e9f0f58 req-f28fc62a-9908-4e43-bfbd-a2c7af8632ad service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:00.774375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-1dd11762-de16-4dbd-a4d4-1c5c3e9f0f58 req-f28fc62a-9908-4e43-bfbd-a2c7af8632ad service nova] Creating event network-vif-deleted:c62c0fe9-fd78-4a72-aef8-9bd42e6d81de for instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 on hv-cinder-maste Jul 11 14:44:00.779947 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-1dd11762-de16-4dbd-a4d4-1c5c3e9f0f58 req-f28fc62a-9908-4e43-bfbd-a2c7af8632ad service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055612 Jul 11 14:44:00.780834 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 282/548] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:44:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:01.216297 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-10838e98-f10f-4f68-9757-cd08a4888c71 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:01.217832 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-10838e98-f10f-4f68-9757-cd08a4888c71 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:01.225455 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-10838e98-f10f-4f68-9757-cd08a4888c71 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:01.226105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-10838e98-f10f-4f68-9757-cd08a4888c71 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:01.227268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-10838e98-f10f-4f68-9757-cd08a4888c71 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:01.227865 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-10838e98-f10f-4f68-9757-cd08a4888c71 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:01.390080 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-10838e98-f10f-4f68-9757-cd08a4888c71 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:01.391462 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-5a07782c-a791-492f-beb6-937efe60ae35 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:01.392179 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-5a07782c-a791-492f-beb6-937efe60ae35 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:01.399779 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5a07782c-a791-492f-beb6-937efe60ae35 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:01.400518 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5a07782c-a791-492f-beb6-937efe60ae35 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:01.400662 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-10838e98-f10f-4f68-9757-cd08a4888c71 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1676 microversion: 2.1 time: 0.187891 Jul 11 14:44:01.401216 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 267/549] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:01 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1676 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:01.401390 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5a07782c-a791-492f-beb6-937efe60ae35 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:01.402082 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5a07782c-a791-492f-beb6-937efe60ae35 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:01.450720 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5a07782c-a791-492f-beb6-937efe60ae35 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:01.451438 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5a07782c-a791-492f-beb6-937efe60ae35 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:01.452056 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.objects.instance [None req-5a07782c-a791-492f-beb6-937efe60ae35 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lazy-loading 'fault' on Instance uuid 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 {{(pid=130159) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:44:01.502798 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-5a07782c-a791-492f-beb6-937efe60ae35 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:01.511264 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-5a07782c-a791-492f-beb6-937efe60ae35 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 200 len: 1332 microversion: 2.1 time: 0.125592 Jul 11 14:44:01.511843 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 283/550] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:01 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 1332 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:02.208325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3d100944-2445-47ee-a231-c9d23b60ead5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c528a5d6-d787-4620-9e4d-83555a31a506"}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:44:02.209458 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-3d100944-2445-47ee-a231-c9d23b60ead5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:02.215807 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d100944-2445-47ee-a231-c9d23b60ead5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:02.216185 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d100944-2445-47ee-a231-c9d23b60ead5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:02.216938 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d100944-2445-47ee-a231-c9d23b60ead5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:02.217470 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d100944-2445-47ee-a231-c9d23b60ead5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:02.358140 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:44:02.418930 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-20b63e83-c1b5-4f2c-8f38-018f867b3276 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:02.420274 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-20b63e83-c1b5-4f2c-8f38-018f867b3276 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:02.422807 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:INSERT=1,nova_api:SELECT=92,nova_api:UPDATE=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:44:02.428735 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-20b63e83-c1b5-4f2c-8f38-018f867b3276 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:02.429462 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-20b63e83-c1b5-4f2c-8f38-018f867b3276 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:02.430446 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-20b63e83-c1b5-4f2c-8f38-018f867b3276 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:02.431045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-20b63e83-c1b5-4f2c-8f38-018f867b3276 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:02.480673 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-3d100944-2445-47ee-a231-c9d23b60ead5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "POST /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.277178 Jul 11 14:44:02.481375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 268/551] 10.222.0.72 () {60 vars in 1322 bytes} [Mon Jul 11 14:44:02 2022] POST /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/os-volume_attachments => generated 194 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:02.530795 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-02d7128a-5789-4219-add5-03dcff8d4e96 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:02.531772 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-02d7128a-5789-4219-add5-03dcff8d4e96 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:02.545497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-02d7128a-5789-4219-add5-03dcff8d4e96 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:02.546177 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-02d7128a-5789-4219-add5-03dcff8d4e96 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:02.547515 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-02d7128a-5789-4219-add5-03dcff8d4e96 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:02.548121 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-02d7128a-5789-4219-add5-03dcff8d4e96 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:02.567862 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-02d7128a-5789-4219-add5-03dcff8d4e96 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] HTTP exception thrown: Instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 could not be found. Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-02d7128a-5789-4219-add5-03dcff8d4e96 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Request method failure captured: Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 HTTP/1.1 Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDZbQUJdl7M18phk4FM9-GGzi8XbGx24YhEK9KsOfI165n5v27J9wq7wzJjs1m7qkLCobj_5JRaSFQlNcVRL5jfwSqdoh1nodKIJxEOpM9gSJP97YpvVDnXLPsMNjwZQwG04QTf0uG5zLXZ7eKV_ivBXCJkCsdmhsPnE9ln-9M7wHs4 Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:44:02.569759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:44:02.571514 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}] Jul 11 14:44:02.571514 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:44:02.571514 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:44:02.571514 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:44:02.571514 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:44:02.571514 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:44:02.571514 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:44:02.571514 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 3ae56fd1172f4bf8806eefc6e8877fef Jul 11 14:44:02.571514 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:44:02.571514 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:44:02.571514 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 could not be found. Jul 11 14:44:02.571514 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': '7a49d1ca-7491-41e0-b0d8-37c92b9d3562'} Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 could not be found. Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:44:02.572367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:44:02.574408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:44:02.574408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:44:02.574408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:44:02.574408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:44:02.574408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:44:02.574408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:44:02.574408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:44:02.574408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 could not be found. Jul 11 14:44:02.574408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:44:02.574408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:44:02.574408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:44:02.574408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:44:02.574408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:44:02.574408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:44:02.574408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:44:02.574408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:44:02.574408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 could not be found. Jul 11 14:44:02.574408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:44:02.577771 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-02d7128a-5789-4219-add5-03dcff8d4e96 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Returning 404 to user: Instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:44:02.578722 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-02d7128a-5789-4219-add5-03dcff8d4e96 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 404 len: 111 microversion: 2.1 time: 0.053688 Jul 11 14:44:02.579254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 269/552] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:02 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 111 bytes in 56 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 11 14:44:02.589709 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f4e6fda9-72db-4122-b152-d69dc7d5f088 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '.version_select at 0x7f59301ca4c0>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:02.596101 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-20b63e83-c1b5-4f2c-8f38-018f867b3276 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:02.609396 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-20b63e83-c1b5-4f2c-8f38-018f867b3276 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1676 microversion: 2.1 time: 0.194861 Jul 11 14:44:02.610675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 284/553] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:02 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1676 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:44:02.615893 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f4e6fda9-72db-4122-b152-d69dc7d5f088 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1046205418" status: 202 len: 0 microversion: 2.1 time: 0.031049 Jul 11 14:44:02.617068 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 270/554] 10.222.0.72 () {58 vars in 1220 bytes} [Mon Jul 11 14:44:02 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1046205418 => generated 0 bytes in 33 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 11 14:44:03.315158 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-369fd452-f919-42fa-a121-d5052c95f152 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:03.315655 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-369fd452-f919-42fa-a121-d5052c95f152 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:03.322394 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-369fd452-f919-42fa-a121-d5052c95f152 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:03.322885 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-369fd452-f919-42fa-a121-d5052c95f152 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:03.323652 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-369fd452-f919-42fa-a121-d5052c95f152 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:03.324093 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-369fd452-f919-42fa-a121-d5052c95f152 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:03.407366 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.objects.instance [None req-369fd452-f919-42fa-a121-d5052c95f152 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lazy-loading 'services' on Instance uuid bd855cba-6708-47de-826b-00c5a74b23f2 {{(pid=130159) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:44:03.482217 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-369fd452-f919-42fa-a121-d5052c95f152 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "DELETE /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments/69ca4bc6-3d1c-4bef-ae85-7e63c008f48d" status: 202 len: 0 microversion: 2.1 time: 0.170826 Jul 11 14:44:03.483061 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 285/555] 10.222.0.72 () {58 vars in 1415 bytes} [Mon Jul 11 14:44:03 2022] DELETE /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments/69ca4bc6-3d1c-4bef-ae85-7e63c008f48d => generated 0 bytes in 172 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:44:03.491357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a0a69552-e930-45cd-8ed1-26be35a81599 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:03.493001 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a0a69552-e930-45cd-8ed1-26be35a81599 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:03.501303 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a0a69552-e930-45cd-8ed1-26be35a81599 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:03.501809 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a0a69552-e930-45cd-8ed1-26be35a81599 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:03.502743 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a0a69552-e930-45cd-8ed1-26be35a81599 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:03.503187 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a0a69552-e930-45cd-8ed1-26be35a81599 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:03.535781 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a0a69552-e930-45cd-8ed1-26be35a81599 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048034 Jul 11 14:44:03.536379 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 271/556] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:44:03 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:03.626849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-31ae0af3-c86b-436a-96d5-81b30708f0b9 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:03.627565 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-31ae0af3-c86b-436a-96d5-81b30708f0b9 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:03.634205 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-31ae0af3-c86b-436a-96d5-81b30708f0b9 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:03.634695 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-31ae0af3-c86b-436a-96d5-81b30708f0b9 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:03.635482 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-31ae0af3-c86b-436a-96d5-81b30708f0b9 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:03.635926 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-31ae0af3-c86b-436a-96d5-81b30708f0b9 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:03.769852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-31ae0af3-c86b-436a-96d5-81b30708f0b9 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:03.777929 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-31ae0af3-c86b-436a-96d5-81b30708f0b9 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1676 microversion: 2.1 time: 0.155422 Jul 11 14:44:03.778536 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 286/557] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:03 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1676 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:04.548377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9c56c0e3-71e8-4a01-8a9f-188917aba24b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:04.549333 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-9c56c0e3-71e8-4a01-8a9f-188917aba24b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:04.556521 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9c56c0e3-71e8-4a01-8a9f-188917aba24b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:04.556974 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9c56c0e3-71e8-4a01-8a9f-188917aba24b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:04.557734 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9c56c0e3-71e8-4a01-8a9f-188917aba24b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:04.558173 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9c56c0e3-71e8-4a01-8a9f-188917aba24b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:04.587203 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9c56c0e3-71e8-4a01-8a9f-188917aba24b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042842 Jul 11 14:44:04.587756 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 272/558] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:44:04 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:04.794646 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-96458e20-f5f2-4a0a-bf8c-1baa266af9cb tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:04.795262 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-96458e20-f5f2-4a0a-bf8c-1baa266af9cb tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:04.801768 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-96458e20-f5f2-4a0a-bf8c-1baa266af9cb tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:04.802157 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-96458e20-f5f2-4a0a-bf8c-1baa266af9cb tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:04.802951 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-96458e20-f5f2-4a0a-bf8c-1baa266af9cb tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:04.803393 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-96458e20-f5f2-4a0a-bf8c-1baa266af9cb tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:04.937180 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-96458e20-f5f2-4a0a-bf8c-1baa266af9cb tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:04.944715 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-96458e20-f5f2-4a0a-bf8c-1baa266af9cb tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1676 microversion: 2.1 time: 0.153956 Jul 11 14:44:04.945320 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 287/559] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:04 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1676 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:44:05.599205 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-39a6e417-1804-417f-8805-baa9498f750e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:05.600129 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-39a6e417-1804-417f-8805-baa9498f750e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:05.606648 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-39a6e417-1804-417f-8805-baa9498f750e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:05.607148 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-39a6e417-1804-417f-8805-baa9498f750e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:05.607886 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-39a6e417-1804-417f-8805-baa9498f750e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:05.608315 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-39a6e417-1804-417f-8805-baa9498f750e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:05.637292 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-39a6e417-1804-417f-8805-baa9498f750e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042023 Jul 11 14:44:05.637843 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 273/560] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:44:05 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:05.962798 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1800b409-620a-47fb-96b4-35fc6c44e4dd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:05.963585 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1800b409-620a-47fb-96b4-35fc6c44e4dd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:05.970500 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1800b409-620a-47fb-96b4-35fc6c44e4dd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:05.970953 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1800b409-620a-47fb-96b4-35fc6c44e4dd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:05.971724 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1800b409-620a-47fb-96b4-35fc6c44e4dd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:05.972159 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1800b409-620a-47fb-96b4-35fc6c44e4dd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:06.099856 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1800b409-620a-47fb-96b4-35fc6c44e4dd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:06.107287 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1800b409-620a-47fb-96b4-35fc6c44e4dd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1676 microversion: 2.1 time: 0.149154 Jul 11 14:44:06.107858 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 288/561] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:05 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1676 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:06.649262 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a258a021-c910-43ea-a5d6-39bfb63f5da8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:06.650147 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a258a021-c910-43ea-a5d6-39bfb63f5da8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:06.656656 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a258a021-c910-43ea-a5d6-39bfb63f5da8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:06.657107 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a258a021-c910-43ea-a5d6-39bfb63f5da8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:06.657873 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a258a021-c910-43ea-a5d6-39bfb63f5da8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:06.658316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a258a021-c910-43ea-a5d6-39bfb63f5da8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:06.688555 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a258a021-c910-43ea-a5d6-39bfb63f5da8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043338 Jul 11 14:44:06.689113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 274/562] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:44:06 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:07.126284 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-ec213f65-9872-4cc1-90e5-2fbbc86bef29 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:07.126869 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-ec213f65-9872-4cc1-90e5-2fbbc86bef29 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:07.134048 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ec213f65-9872-4cc1-90e5-2fbbc86bef29 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:07.134529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ec213f65-9872-4cc1-90e5-2fbbc86bef29 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:07.135305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ec213f65-9872-4cc1-90e5-2fbbc86bef29 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:07.135744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ec213f65-9872-4cc1-90e5-2fbbc86bef29 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:07.262704 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-ec213f65-9872-4cc1-90e5-2fbbc86bef29 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:07.270198 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-ec213f65-9872-4cc1-90e5-2fbbc86bef29 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1676 microversion: 2.1 time: 0.147957 Jul 11 14:44:07.270801 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 289/563] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:07 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1676 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:07.700058 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-95cebdec-606c-4a3c-b823-9ed1591b646c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:07.701058 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-95cebdec-606c-4a3c-b823-9ed1591b646c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:07.707840 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-95cebdec-606c-4a3c-b823-9ed1591b646c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:07.708289 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-95cebdec-606c-4a3c-b823-9ed1591b646c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:07.709043 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-95cebdec-606c-4a3c-b823-9ed1591b646c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:07.709486 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-95cebdec-606c-4a3c-b823-9ed1591b646c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:07.740191 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-95cebdec-606c-4a3c-b823-9ed1591b646c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043976 Jul 11 14:44:07.740732 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 275/564] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:44:07 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:44:08.287571 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2c5d5687-f852-4fe9-b3f5-2e926d77ea8a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:08.288169 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2c5d5687-f852-4fe9-b3f5-2e926d77ea8a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:08.294909 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2c5d5687-f852-4fe9-b3f5-2e926d77ea8a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:08.295373 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2c5d5687-f852-4fe9-b3f5-2e926d77ea8a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:08.296137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2c5d5687-f852-4fe9-b3f5-2e926d77ea8a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:08.296581 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2c5d5687-f852-4fe9-b3f5-2e926d77ea8a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:08.425050 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-2c5d5687-f852-4fe9-b3f5-2e926d77ea8a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:08.433171 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2c5d5687-f852-4fe9-b3f5-2e926d77ea8a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1676 microversion: 2.1 time: 0.149420 Jul 11 14:44:08.433737 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 290/565] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:08 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1676 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:08.753861 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2a59eda3-ed17-4065-b85f-542a5974bcb9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:08.754907 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2a59eda3-ed17-4065-b85f-542a5974bcb9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:08.761423 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2a59eda3-ed17-4065-b85f-542a5974bcb9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:08.761870 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2a59eda3-ed17-4065-b85f-542a5974bcb9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:08.762689 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2a59eda3-ed17-4065-b85f-542a5974bcb9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:08.763127 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2a59eda3-ed17-4065-b85f-542a5974bcb9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:08.791945 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2a59eda3-ed17-4065-b85f-542a5974bcb9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042006 Jul 11 14:44:08.792424 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 276/566] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:44:08 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:09.450090 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-ecd771b6-396a-42bb-94f7-bf78966be931 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:09.450851 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-ecd771b6-396a-42bb-94f7-bf78966be931 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:09.457731 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ecd771b6-396a-42bb-94f7-bf78966be931 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:09.458185 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ecd771b6-396a-42bb-94f7-bf78966be931 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:09.458985 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ecd771b6-396a-42bb-94f7-bf78966be931 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:09.459427 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ecd771b6-396a-42bb-94f7-bf78966be931 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:09.588942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-ecd771b6-396a-42bb-94f7-bf78966be931 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:09.596611 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-ecd771b6-396a-42bb-94f7-bf78966be931 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1676 microversion: 2.1 time: 0.150403 Jul 11 14:44:09.597168 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 291/567] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:09 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1676 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:09.803495 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2671e291-44fa-4408-b2d7-7e3688d05fa7 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:09.804423 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2671e291-44fa-4408-b2d7-7e3688d05fa7 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:09.811803 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2671e291-44fa-4408-b2d7-7e3688d05fa7 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:09.812249 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2671e291-44fa-4408-b2d7-7e3688d05fa7 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:09.813002 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2671e291-44fa-4408-b2d7-7e3688d05fa7 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:09.813438 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2671e291-44fa-4408-b2d7-7e3688d05fa7 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:09.848023 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2671e291-44fa-4408-b2d7-7e3688d05fa7 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048473 Jul 11 14:44:09.848573 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 277/568] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:44:09 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:10.614062 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-4f332c16-88a2-492b-befd-48f6976026e6 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:10.614841 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-4f332c16-88a2-492b-befd-48f6976026e6 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:10.616801 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=2 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:44:10.622586 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4f332c16-88a2-492b-befd-48f6976026e6 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:10.623043 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4f332c16-88a2-492b-befd-48f6976026e6 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:10.623797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4f332c16-88a2-492b-befd-48f6976026e6 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:10.624230 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4f332c16-88a2-492b-befd-48f6976026e6 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:10.767123 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-4f332c16-88a2-492b-befd-48f6976026e6 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:10.778335 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-4f332c16-88a2-492b-befd-48f6976026e6 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1653 microversion: 2.1 time: 0.167377 Jul 11 14:44:10.778591 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 292/569] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:10 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1653 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:10.794241 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-b926dadc-0854-4c48-8120-efe5f7faad41 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:44:10.796393 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=97,nova_api:DELETE=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:44:10.804509 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b926dadc-0854-4c48-8120-efe5f7faad41 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:10.805452 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b926dadc-0854-4c48-8120-efe5f7faad41 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:10.828890 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-b926dadc-0854-4c48-8120-efe5f7faad41 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Going to try to start instance {{(pid=130160) start /opt/stack/nova/nova/compute/api.py:2730}} Jul 11 14:44:10.860632 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-cd384d2d-21ce-4bfc-b4fc-4e9f7f3a6ddf tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:10.861607 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-cd384d2d-21ce-4bfc-b4fc-4e9f7f3a6ddf tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:10.867801 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cd384d2d-21ce-4bfc-b4fc-4e9f7f3a6ddf tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:10.868361 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cd384d2d-21ce-4bfc-b4fc-4e9f7f3a6ddf tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:10.869185 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cd384d2d-21ce-4bfc-b4fc-4e9f7f3a6ddf tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:10.869701 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cd384d2d-21ce-4bfc-b4fc-4e9f7f3a6ddf tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:10.872453 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-b926dadc-0854-4c48-8120-efe5f7faad41 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "POST /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d/action" status: 202 len: 0 microversion: 2.1 time: 0.081938 Jul 11 14:44:10.873095 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 278/570] 10.222.0.72 () {60 vars in 1277 bytes} [Mon Jul 11 14:44:10 2022] POST /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d/action => generated 0 bytes in 83 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:44:10.881752 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9cc61558-8c9c-4d7f-815b-b26d51e07ee8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:10.883449 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-9cc61558-8c9c-4d7f-815b-b26d51e07ee8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:10.889457 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9cc61558-8c9c-4d7f-815b-b26d51e07ee8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:10.889896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9cc61558-8c9c-4d7f-815b-b26d51e07ee8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:10.890823 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9cc61558-8c9c-4d7f-815b-b26d51e07ee8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:10.891553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9cc61558-8c9c-4d7f-815b-b26d51e07ee8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:10.902638 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-cd384d2d-21ce-4bfc-b4fc-4e9f7f3a6ddf tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046045 Jul 11 14:44:10.903640 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 293/571] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:44:10 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:44:11.040439 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-9cc61558-8c9c-4d7f-815b-b26d51e07ee8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:11.048196 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9cc61558-8c9c-4d7f-815b-b26d51e07ee8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1662 microversion: 2.1 time: 0.170619 Jul 11 14:44:11.056939 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 279/572] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:10 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1662 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:44:11.915061 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-0f1b0b2f-75a5-46bf-8806-d55cc25c061c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:11.915978 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-0f1b0b2f-75a5-46bf-8806-d55cc25c061c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:11.922660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0f1b0b2f-75a5-46bf-8806-d55cc25c061c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:11.923113 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0f1b0b2f-75a5-46bf-8806-d55cc25c061c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:11.923874 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0f1b0b2f-75a5-46bf-8806-d55cc25c061c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:11.924381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0f1b0b2f-75a5-46bf-8806-d55cc25c061c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:11.954349 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-0f1b0b2f-75a5-46bf-8806-d55cc25c061c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043850 Jul 11 14:44:11.954965 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 294/573] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:44:11 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:12.065105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-cc4a22a2-8398-483e-98e9-bc4417418608 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:12.065797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-cc4a22a2-8398-483e-98e9-bc4417418608 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:12.072888 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc4a22a2-8398-483e-98e9-bc4417418608 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:12.073340 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc4a22a2-8398-483e-98e9-bc4417418608 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:12.074131 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc4a22a2-8398-483e-98e9-bc4417418608 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:12.074611 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc4a22a2-8398-483e-98e9-bc4417418608 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:12.208321 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-cc4a22a2-8398-483e-98e9-bc4417418608 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:12.216984 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-cc4a22a2-8398-483e-98e9-bc4417418608 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1662 microversion: 2.1 time: 0.155738 Jul 11 14:44:12.217551 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 280/574] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:12 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1662 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:12.966993 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-791e1082-8f15-4cd2-ba43-af3ab32afe9e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:12.968014 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-791e1082-8f15-4cd2-ba43-af3ab32afe9e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:12.974613 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-791e1082-8f15-4cd2-ba43-af3ab32afe9e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:12.975064 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-791e1082-8f15-4cd2-ba43-af3ab32afe9e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:12.975822 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-791e1082-8f15-4cd2-ba43-af3ab32afe9e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:12.976259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-791e1082-8f15-4cd2-ba43-af3ab32afe9e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:13.004865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-791e1082-8f15-4cd2-ba43-af3ab32afe9e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041923 Jul 11 14:44:13.005417 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 295/575] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:44:12 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:13.234138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-cf6c7a1d-feb5-4dd3-84a8-9b7e2d480c11 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:13.234692 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-cf6c7a1d-feb5-4dd3-84a8-9b7e2d480c11 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:13.241318 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cf6c7a1d-feb5-4dd3-84a8-9b7e2d480c11 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:13.241768 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cf6c7a1d-feb5-4dd3-84a8-9b7e2d480c11 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:13.243103 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cf6c7a1d-feb5-4dd3-84a8-9b7e2d480c11 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:13.243499 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cf6c7a1d-feb5-4dd3-84a8-9b7e2d480c11 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:13.369459 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-cf6c7a1d-feb5-4dd3-84a8-9b7e2d480c11 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:13.376930 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-cf6c7a1d-feb5-4dd3-84a8-9b7e2d480c11 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1662 microversion: 2.1 time: 0.146765 Jul 11 14:44:13.377494 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 281/576] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:13 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1662 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:14.017345 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-899de69c-35b4-4855-a1bd-06866036610a tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:14.018264 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-899de69c-35b4-4855-a1bd-06866036610a tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:14.025555 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-899de69c-35b4-4855-a1bd-06866036610a tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:14.026004 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-899de69c-35b4-4855-a1bd-06866036610a tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:14.026801 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-899de69c-35b4-4855-a1bd-06866036610a tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:14.027235 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-899de69c-35b4-4855-a1bd-06866036610a tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:14.053878 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:44:14.058809 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-899de69c-35b4-4855-a1bd-06866036610a tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045546 Jul 11 14:44:14.059603 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 296/577] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:44:14 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:14.394885 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-876325db-7b3d-499c-958d-2ea501a11563 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:14.395434 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-876325db-7b3d-499c-958d-2ea501a11563 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:14.402540 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-876325db-7b3d-499c-958d-2ea501a11563 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:14.402994 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-876325db-7b3d-499c-958d-2ea501a11563 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:14.403766 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-876325db-7b3d-499c-958d-2ea501a11563 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:14.404203 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-876325db-7b3d-499c-958d-2ea501a11563 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:14.536687 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-876325db-7b3d-499c-958d-2ea501a11563 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:14.544528 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-876325db-7b3d-499c-958d-2ea501a11563 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1662 microversion: 2.1 time: 0.153573 Jul 11 14:44:14.545110 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 282/578] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:14 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1662 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:15.071936 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-ec4411b0-10c4-4a58-8041-7f2abc144883 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:15.072854 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-ec4411b0-10c4-4a58-8041-7f2abc144883 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:15.080696 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ec4411b0-10c4-4a58-8041-7f2abc144883 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:15.081150 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ec4411b0-10c4-4a58-8041-7f2abc144883 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:15.082186 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ec4411b0-10c4-4a58-8041-7f2abc144883 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:15.082656 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ec4411b0-10c4-4a58-8041-7f2abc144883 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:15.111972 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-ec4411b0-10c4-4a58-8041-7f2abc144883 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045053 Jul 11 14:44:15.112445 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 297/579] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:44:15 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:15.561075 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-316df7aa-7b33-480c-834d-0f18f29faad9 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:15.561823 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-316df7aa-7b33-480c-834d-0f18f29faad9 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:15.568201 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-316df7aa-7b33-480c-834d-0f18f29faad9 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:15.568645 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-316df7aa-7b33-480c-834d-0f18f29faad9 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:15.569394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-316df7aa-7b33-480c-834d-0f18f29faad9 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:15.569826 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-316df7aa-7b33-480c-834d-0f18f29faad9 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:15.708592 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-316df7aa-7b33-480c-834d-0f18f29faad9 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:15.716434 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-316df7aa-7b33-480c-834d-0f18f29faad9 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1666 microversion: 2.1 time: 0.159228 Jul 11 14:44:15.716989 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 283/580] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:15 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1666 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:16.123538 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-85d94a55-536f-44a3-bc7a-fcf17dd8af25 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:16.124453 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-85d94a55-536f-44a3-bc7a-fcf17dd8af25 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:16.131043 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-85d94a55-536f-44a3-bc7a-fcf17dd8af25 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:16.131494 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-85d94a55-536f-44a3-bc7a-fcf17dd8af25 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:16.132251 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-85d94a55-536f-44a3-bc7a-fcf17dd8af25 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:16.133207 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-85d94a55-536f-44a3-bc7a-fcf17dd8af25 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:16.162317 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-85d94a55-536f-44a3-bc7a-fcf17dd8af25 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042739 Jul 11 14:44:16.162909 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 298/581] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:44:16 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:16.480296 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-da4150d8-d06f-4059-9125-08efe239675a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2be6be9e-10da-4656-a426-f2a29458eb3c"}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:44:16.481273 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-da4150d8-d06f-4059-9125-08efe239675a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:16.487240 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-da4150d8-d06f-4059-9125-08efe239675a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:16.487695 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-da4150d8-d06f-4059-9125-08efe239675a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:16.488477 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-da4150d8-d06f-4059-9125-08efe239675a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:16.488905 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-da4150d8-d06f-4059-9125-08efe239675a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:16.645866 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:44:16.761632 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-da4150d8-d06f-4059-9125-08efe239675a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "POST /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.285291 Jul 11 14:44:16.762274 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 284/582] 10.222.0.72 () {60 vars in 1322 bytes} [Mon Jul 11 14:44:16 2022] POST /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-volume_attachments => generated 194 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:17.176509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e62ef159-cc7d-40cf-9cb6-a338e5299a43 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:17.180387 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e62ef159-cc7d-40cf-9cb6-a338e5299a43 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:17.195005 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e62ef159-cc7d-40cf-9cb6-a338e5299a43 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:17.195432 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e62ef159-cc7d-40cf-9cb6-a338e5299a43 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:17.196278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e62ef159-cc7d-40cf-9cb6-a338e5299a43 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:17.196866 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e62ef159-cc7d-40cf-9cb6-a338e5299a43 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:17.236298 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e62ef159-cc7d-40cf-9cb6-a338e5299a43 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066564 Jul 11 14:44:17.236864 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 299/583] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:44:17 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:17.422781 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-51b2efa2-9777-4bd4-8d28-bd82ba26a483 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:17.423470 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-51b2efa2-9777-4bd4-8d28-bd82ba26a483 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:17.429899 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-51b2efa2-9777-4bd4-8d28-bd82ba26a483 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:17.430342 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-51b2efa2-9777-4bd4-8d28-bd82ba26a483 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:17.431344 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-51b2efa2-9777-4bd4-8d28-bd82ba26a483 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:17.431900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-51b2efa2-9777-4bd4-8d28-bd82ba26a483 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:17.574735 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-51b2efa2-9777-4bd4-8d28-bd82ba26a483 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:17.584366 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-51b2efa2-9777-4bd4-8d28-bd82ba26a483 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1592 microversion: 2.1 time: 0.166259 Jul 11 14:44:17.584948 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 285/584] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:17 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1592 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:18.249063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-42b6e77e-50ad-4f52-8697-887b4db1b24c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:18.249819 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-79c43461-8fc5-4655-a908-ed60b5df5709 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:18.250251 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-42b6e77e-50ad-4f52-8697-887b4db1b24c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:18.250959 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-79c43461-8fc5-4655-a908-ed60b5df5709 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:18.257039 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-42b6e77e-50ad-4f52-8697-887b4db1b24c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:18.257528 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-42b6e77e-50ad-4f52-8697-887b4db1b24c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:18.258074 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-79c43461-8fc5-4655-a908-ed60b5df5709 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:18.258372 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-42b6e77e-50ad-4f52-8697-887b4db1b24c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:18.258708 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-79c43461-8fc5-4655-a908-ed60b5df5709 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:18.259242 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-42b6e77e-50ad-4f52-8697-887b4db1b24c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:18.259722 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-79c43461-8fc5-4655-a908-ed60b5df5709 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:18.260219 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-79c43461-8fc5-4655-a908-ed60b5df5709 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:18.291693 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-42b6e77e-50ad-4f52-8697-887b4db1b24c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046713 Jul 11 14:44:18.292273 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 300/585] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:44:18 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 11 14:44:18.360614 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f233ef41-d6cd-402e-a605-a58f816949e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:18.361118 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f233ef41-d6cd-402e-a605-a58f816949e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:18.367833 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f233ef41-d6cd-402e-a605-a58f816949e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:18.368289 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f233ef41-d6cd-402e-a605-a58f816949e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:18.369064 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f233ef41-d6cd-402e-a605-a58f816949e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:18.369508 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f233ef41-d6cd-402e-a605-a58f816949e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:18.395542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f233ef41-d6cd-402e-a605-a58f816949e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Going to try to terminate instance {{(pid=130159) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:44:18.417129 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.objects.instance [None req-79c43461-8fc5-4655-a908-ed60b5df5709 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lazy-loading 'services' on Instance uuid 7e398a3c-0f26-422e-8cb2-828013595327 {{(pid=130160) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:44:18.513379 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f233ef41-d6cd-402e-a605-a58f816949e0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "DELETE /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 204 len: 0 microversion: 2.1 time: 0.157541 Jul 11 14:44:18.517086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 301/586] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:44:18 2022] DELETE /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 11 14:44:18.529332 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9d33498f-d160-40e8-accf-b8a0508b2193 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:18.530287 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9d33498f-d160-40e8-accf-b8a0508b2193 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:18.537904 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9d33498f-d160-40e8-accf-b8a0508b2193 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:18.538350 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9d33498f-d160-40e8-accf-b8a0508b2193 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:18.539886 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9d33498f-d160-40e8-accf-b8a0508b2193 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:18.547059 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9d33498f-d160-40e8-accf-b8a0508b2193 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:18.557697 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-79c43461-8fc5-4655-a908-ed60b5df5709 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "DELETE /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327/os-volume_attachments/593b97e2-90cf-4784-a426-92970e597a12" status: 202 len: 0 microversion: 2.1 time: 0.311740 Jul 11 14:44:18.558362 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 286/587] 10.222.0.72 () {58 vars in 1415 bytes} [Mon Jul 11 14:44:18 2022] DELETE /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327/os-volume_attachments/593b97e2-90cf-4784-a426-92970e597a12 => generated 0 bytes in 314 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:44:18.696128 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9d33498f-d160-40e8-accf-b8a0508b2193 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:18.704176 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9d33498f-d160-40e8-accf-b8a0508b2193 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1682 microversion: 2.1 time: 0.182187 Jul 11 14:44:18.704841 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 302/588] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:18 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1682 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:19.307490 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-3de1f011-fae6-4fc6-85c1-5d8f1d960861 req-cbf3f295-8d3f-43e8-a276-60be8c902a43 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0033415c-3adf-4fe8-bac6-60cf7774143f", "tag": "eacb6063-c62f-4742-8ca6-7f93d25fdaf8"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:44:19.316329 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-3de1f011-fae6-4fc6-85c1-5d8f1d960861 req-cbf3f295-8d3f-43e8-a276-60be8c902a43 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:19.317154 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-3de1f011-fae6-4fc6-85c1-5d8f1d960861 req-cbf3f295-8d3f-43e8-a276-60be8c902a43 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:19.330320 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-3de1f011-fae6-4fc6-85c1-5d8f1d960861 req-cbf3f295-8d3f-43e8-a276-60be8c902a43 service nova] Creating event network-changed:eacb6063-c62f-4742-8ca6-7f93d25fdaf8 for instance 0033415c-3adf-4fe8-bac6-60cf7774143f on hv-cinder-maste Jul 11 14:44:19.335918 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-3de1f011-fae6-4fc6-85c1-5d8f1d960861 req-cbf3f295-8d3f-43e8-a276-60be8c902a43 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033715 Jul 11 14:44:19.336734 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 287/589] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:44:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:19.722341 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9d2c4e28-9ccb-4310-a332-b1f3416179d3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:19.723012 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9d2c4e28-9ccb-4310-a332-b1f3416179d3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:19.733011 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9d2c4e28-9ccb-4310-a332-b1f3416179d3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:19.733464 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9d2c4e28-9ccb-4310-a332-b1f3416179d3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:19.734234 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9d2c4e28-9ccb-4310-a332-b1f3416179d3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:19.734707 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9d2c4e28-9ccb-4310-a332-b1f3416179d3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:19.881489 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9d2c4e28-9ccb-4310-a332-b1f3416179d3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:19.889116 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9d2c4e28-9ccb-4310-a332-b1f3416179d3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1682 microversion: 2.1 time: 0.170655 Jul 11 14:44:19.889915 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 303/590] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:19 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1682 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:20.263352 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2bb5b595-f5b8-4a3c-ab2a-8c4cbfae557f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-1911000918"}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:44:20.263973 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2bb5b595-f5b8-4a3c-ab2a-8c4cbfae557f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:20.270563 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2bb5b595-f5b8-4a3c-ab2a-8c4cbfae557f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:20.271013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2bb5b595-f5b8-4a3c-ab2a-8c4cbfae557f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:20.271777 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2bb5b595-f5b8-4a3c-ab2a-8c4cbfae557f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:20.272207 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2bb5b595-f5b8-4a3c-ab2a-8c4cbfae557f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:20.406121 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.network.security_group_api [None req-2bb5b595-f5b8-4a3c-ab2a-8c4cbfae557f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Adding security group 65fd9fc5-5855-468e-9322-c09a03998b07 to port eacb6063-c62f-4742-8ca6-7f93d25fdaf8 Jul 11 14:44:20.906155 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-5997dabb-3b18-4eba-a9f3-37e0c21c68bd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:20.906883 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-5997dabb-3b18-4eba-a9f3-37e0c21c68bd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:20.913473 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5997dabb-3b18-4eba-a9f3-37e0c21c68bd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:20.913954 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5997dabb-3b18-4eba-a9f3-37e0c21c68bd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:20.914739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5997dabb-3b18-4eba-a9f3-37e0c21c68bd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:20.915171 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5997dabb-3b18-4eba-a9f3-37e0c21c68bd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:20.977396 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2bb5b595-f5b8-4a3c-ab2a-8c4cbfae557f tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "POST /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/action" status: 202 len: 0 microversion: 2.1 time: 0.718754 Jul 11 14:44:20.981680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 288/591] 10.222.0.72 () {60 vars in 1277 bytes} [Mon Jul 11 14:44:20 2022] POST /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/action => generated 0 bytes in 724 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 11 14:44:20.988014 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-bf9b6409-0846-45e8-9c7d-f594f3968b6a tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:20.988834 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-bf9b6409-0846-45e8-9c7d-f594f3968b6a tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:21.002402 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bf9b6409-0846-45e8-9c7d-f594f3968b6a tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:21.002890 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bf9b6409-0846-45e8-9c7d-f594f3968b6a tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:21.003698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bf9b6409-0846-45e8-9c7d-f594f3968b6a tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:21.004132 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bf9b6409-0846-45e8-9c7d-f594f3968b6a tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:21.120101 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-5997dabb-3b18-4eba-a9f3-37e0c21c68bd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:21.128594 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-5997dabb-3b18-4eba-a9f3-37e0c21c68bd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1682 microversion: 2.1 time: 0.226198 Jul 11 14:44:21.140992 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 304/592] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:20 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1682 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:44:21.222684 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-bf9b6409-0846-45e8-9c7d-f594f3968b6a tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:21.233940 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-bf9b6409-0846-45e8-9c7d-f594f3968b6a tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1639 microversion: 2.1 time: 0.248743 Jul 11 14:44:21.234859 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 289/593] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:20 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1639 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:21.365330 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-8e3819ff-8259-41fe-b4c7-d787355cc6a0 req-2e196fe1-1d89-407e-9543-5b739af6f0eb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0033415c-3adf-4fe8-bac6-60cf7774143f", "tag": "eacb6063-c62f-4742-8ca6-7f93d25fdaf8"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:44:21.373609 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-8e3819ff-8259-41fe-b4c7-d787355cc6a0 req-2e196fe1-1d89-407e-9543-5b739af6f0eb service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:21.374422 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-8e3819ff-8259-41fe-b4c7-d787355cc6a0 req-2e196fe1-1d89-407e-9543-5b739af6f0eb service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:21.389669 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-8e3819ff-8259-41fe-b4c7-d787355cc6a0 req-2e196fe1-1d89-407e-9543-5b739af6f0eb service nova] Creating event network-changed:eacb6063-c62f-4742-8ca6-7f93d25fdaf8 for instance 0033415c-3adf-4fe8-bac6-60cf7774143f on hv-cinder-maste Jul 11 14:44:21.395748 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-8e3819ff-8259-41fe-b4c7-d787355cc6a0 req-2e196fe1-1d89-407e-9543-5b739af6f0eb service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033614 Jul 11 14:44:21.396988 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 305/594] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:44:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:22.149289 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e128f715-3dfc-425f-8286-99cba7c41cda tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:22.150921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-e128f715-3dfc-425f-8286-99cba7c41cda tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:22.159589 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e128f715-3dfc-425f-8286-99cba7c41cda tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:22.160047 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e128f715-3dfc-425f-8286-99cba7c41cda tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:22.160830 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e128f715-3dfc-425f-8286-99cba7c41cda tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:22.162006 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e128f715-3dfc-425f-8286-99cba7c41cda tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:22.264204 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e128f715-3dfc-425f-8286-99cba7c41cda tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:22.272122 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e128f715-3dfc-425f-8286-99cba7c41cda tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 200 len: 1614 microversion: 2.1 time: 0.129138 Jul 11 14:44:22.272703 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 290/595] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:22 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 1614 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:23.290859 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d5e7c387-41bd-405d-b045-234981d8c5dc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:23.291836 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d5e7c387-41bd-405d-b045-234981d8c5dc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:23.299708 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d5e7c387-41bd-405d-b045-234981d8c5dc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:23.300315 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d5e7c387-41bd-405d-b045-234981d8c5dc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:23.301371 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d5e7c387-41bd-405d-b045-234981d8c5dc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:23.302024 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d5e7c387-41bd-405d-b045-234981d8c5dc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:23.316387 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-d5e7c387-41bd-405d-b045-234981d8c5dc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] HTTP exception thrown: Instance bd855cba-6708-47de-826b-00c5a74b23f2 could not be found. Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d5e7c387-41bd-405d-b045-234981d8c5dc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Request method failure captured: Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 HTTP/1.1 Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZbQUJdl7M18phk4FM9-GGzi8XbGx24YhEK9KsOfI165n5v27J9wq7wzJjs1m7qkLCobj_5JRaSFQlNcVRL5jfwSqdoh1nodKIJxEOpM9gSJP97YpvVDnXLPsMNjwZQwG04QTf0uG5zLXZ7eKV_ivBXCJkCsdmhsPnE9ln-9M7wHs4 Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:44:23.318311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:44:23.320478 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}] Jul 11 14:44:23.320478 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:44:23.320478 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:44:23.320478 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:44:23.320478 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:44:23.320478 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:44:23.320478 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:44:23.320478 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 3ae56fd1172f4bf8806eefc6e8877fef Jul 11 14:44:23.320478 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:44:23.320478 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:44:23.320478 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance bd855cba-6708-47de-826b-00c5a74b23f2 could not be found. Jul 11 14:44:23.320478 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': 'bd855cba-6708-47de-826b-00c5a74b23f2'} Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bd855cba-6708-47de-826b-00c5a74b23f2 could not be found. Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:44:23.321383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:44:23.322848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:44:23.322848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:44:23.322848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:44:23.322848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:44:23.322848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:44:23.322848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:44:23.322848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:44:23.322848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bd855cba-6708-47de-826b-00c5a74b23f2 could not be found. Jul 11 14:44:23.322848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:44:23.322848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:44:23.322848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:44:23.322848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:44:23.322848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:44:23.322848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:44:23.322848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:44:23.322848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:44:23.322848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bd855cba-6708-47de-826b-00c5a74b23f2 could not be found. Jul 11 14:44:23.322848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:44:23.324925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d5e7c387-41bd-405d-b045-234981d8c5dc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Returning 404 to user: Instance bd855cba-6708-47de-826b-00c5a74b23f2 could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:44:23.326012 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-d5e7c387-41bd-405d-b045-234981d8c5dc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 404 len: 111 microversion: 2.1 time: 0.040437 Jul 11 14:44:23.326611 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 306/596] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:23 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:44:23.336938 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c9e6fa4e-f8eb-43ce-bc42-065ba235feef tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '.version_select at 0x7f593017ea60>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:23.360345 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c9e6fa4e-f8eb-43ce-bc42-065ba235feef tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1674144846" status: 202 len: 0 microversion: 2.1 time: 0.028910 Jul 11 14:44:23.360855 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 291/597] 10.222.0.72 () {58 vars in 1220 bytes} [Mon Jul 11 14:44:23 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1674144846 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:44:23.419217 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-63a8a71b-2141-4167-a571-b9e23d1035eb req-79662546-27c1-4bd1-9cf4-2edc03961a24 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bd855cba-6708-47de-826b-00c5a74b23f2", "name": "network-vif-deleted", "tag": "8b901616-9eed-4045-b039-9047df0bd8e7"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:44:23.430536 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-63a8a71b-2141-4167-a571-b9e23d1035eb req-79662546-27c1-4bd1-9cf4-2edc03961a24 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:23.431256 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-63a8a71b-2141-4167-a571-b9e23d1035eb req-79662546-27c1-4bd1-9cf4-2edc03961a24 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:23.447844 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-63a8a71b-2141-4167-a571-b9e23d1035eb req-79662546-27c1-4bd1-9cf4-2edc03961a24 service nova] Creating event network-vif-deleted:8b901616-9eed-4045-b039-9047df0bd8e7 for instance bd855cba-6708-47de-826b-00c5a74b23f2 on hv-cinder-maste Jul 11 14:44:23.454875 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-63a8a71b-2141-4167-a571-b9e23d1035eb req-79662546-27c1-4bd1-9cf4-2edc03961a24 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040442 Jul 11 14:44:23.456850 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 307/598] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:44:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:23.982183 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-b19ffd48-1314-4508-ba1b-a8cfcce1bc8b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Action: 'create', calling method: .version_select at 0x7f59301b68b0>, body: {"keypair": {"name": "tempest-keypair-2057892801", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMQZhFzPRIxyXX6CFU3sYyYIaiS5kafbbvG5NZIoxkcutytC7g/osYYj1j/Y4ix/wF8qgjnCYrRMOcYnu9dq2+PzqJaf33R1TnM0TFstwl3P0dJp/hPoZE25JEwPWs9fsQ=="}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:44:24.002703 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-b19ffd48-1314-4508-ba1b-a8cfcce1bc8b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Getting quotas for project 1baa6ea2e19747d1af1e1042c362895d. Resources: {'key_pairs'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:44:24.006593 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-b19ffd48-1314-4508-ba1b-a8cfcce1bc8b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Getting quotas for user 3ae56fd1172f4bf8806eefc6e8877fef and project 1baa6ea2e19747d1af1e1042c362895d. Resources: {'key_pairs'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:44:24.030653 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-b19ffd48-1314-4508-ba1b-a8cfcce1bc8b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.052209 Jul 11 14:44:24.031258 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 292/599] 10.222.0.72 () {60 vars in 1158 bytes} [Mon Jul 11 14:44:23 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:25.339379 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-ca73ada9-9e3b-4272-b938-8a101a5571e9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-906012423", "imageRef": "a588fa24-23b5-4e53-87c6-7e600ee3a5ef", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8679b039-4e4e-4ae1-83e8-21c78c675a96"}], "security_groups": [{"name": "tempest-securitygroup--1839908876"}], "key_name": "tempest-keypair-2057892801", "user_data": "}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:44:25.415100 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a8894aba-5f4a-401e-9d92-fb6af21d1bfd tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:44:25.416727 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a8894aba-5f4a-401e-9d92-fb6af21d1bfd tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:25.424608 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a8894aba-5f4a-401e-9d92-fb6af21d1bfd tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:25.424806 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a8894aba-5f4a-401e-9d92-fb6af21d1bfd tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:25.425531 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a8894aba-5f4a-401e-9d92-fb6af21d1bfd tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:25.425973 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a8894aba-5f4a-401e-9d92-fb6af21d1bfd tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:25.450757 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.objects.instance [None req-a8894aba-5f4a-401e-9d92-fb6af21d1bfd tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lazy-loading 'flavor' on Instance uuid f66823cd-4fe8-42cf-ab7a-d7cd7afc289a {{(pid=130160) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:44:25.477848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.network.neutron [None req-ca73ada9-9e3b-4272-b938-8a101a5571e9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] validate_networks() for [('8679b039-4e4e-4ae1-83e8-21c78c675a96', None, None, None, None, None)] {{(pid=130159) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:44:25.588415 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a8894aba-5f4a-401e-9d92-fb6af21d1bfd tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "POST /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/action" status: 200 len: 32 microversion: 2.1 time: 0.177826 Jul 11 14:44:25.589200 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 293/600] 10.222.0.72 () {60 vars in 1277 bytes} [Mon Jul 11 14:44:25 2022] POST /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/action => generated 32 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 11 14:44:25.599634 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e750edb7-3857-416d-9d14-09256f3d63a3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:25.600330 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-e750edb7-3857-416d-9d14-09256f3d63a3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:25.607390 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e750edb7-3857-416d-9d14-09256f3d63a3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:25.607836 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e750edb7-3857-416d-9d14-09256f3d63a3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:25.608746 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e750edb7-3857-416d-9d14-09256f3d63a3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:25.609260 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e750edb7-3857-416d-9d14-09256f3d63a3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:25.753150 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e750edb7-3857-416d-9d14-09256f3d63a3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:25.761634 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e750edb7-3857-416d-9d14-09256f3d63a3 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.167375 Jul 11 14:44:25.762218 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 294/601] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:25 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:44:26.022099 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-4172993f-7dab-486f-b0f1-70002c6f6683 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:26.022644 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-4172993f-7dab-486f-b0f1-70002c6f6683 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:26.028841 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4172993f-7dab-486f-b0f1-70002c6f6683 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:26.029289 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4172993f-7dab-486f-b0f1-70002c6f6683 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:26.030054 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4172993f-7dab-486f-b0f1-70002c6f6683 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:26.030509 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4172993f-7dab-486f-b0f1-70002c6f6683 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:26.031129 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-ca73ada9-9e3b-4272-b938-8a101a5571e9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Flavor limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:44:26.031558 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-ca73ada9-9e3b-4272-b938-8a101a5571e9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Image limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:44:26.032032 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-ca73ada9-9e3b-4272-b938-8a101a5571e9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Flavor pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:44:26.032630 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-ca73ada9-9e3b-4272-b938-8a101a5571e9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Image pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:44:26.033241 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-ca73ada9-9e3b-4272-b938-8a101a5571e9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:44:26.042554 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ca73ada9-9e3b-4272-b938-8a101a5571e9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:26.043298 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ca73ada9-9e3b-4272-b938-8a101a5571e9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:26.044113 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ca73ada9-9e3b-4272-b938-8a101a5571e9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:26.044554 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ca73ada9-9e3b-4272-b938-8a101a5571e9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:26.060023 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-4172993f-7dab-486f-b0f1-70002c6f6683 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Going to try to terminate instance {{(pid=130160) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:44:26.064860 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-ca73ada9-9e3b-4272-b938-8a101a5571e9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Getting quotas for project 1baa6ea2e19747d1af1e1042c362895d. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:44:26.069405 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-ca73ada9-9e3b-4272-b938-8a101a5571e9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Getting quotas for user 3ae56fd1172f4bf8806eefc6e8877fef and project 1baa6ea2e19747d1af1e1042c362895d. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:44:26.082128 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-ca73ada9-9e3b-4272-b938-8a101a5571e9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Going to run 1 instances... {{(pid=130159) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:44:26.084606 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-ca73ada9-9e3b-4272-b938-8a101a5571e9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a588fa24-23b5-4e53-87c6-7e600ee3a5ef',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130159) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:44:26.135928 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-ca73ada9-9e3b-4272-b938-8a101a5571e9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.803333 Jul 11 14:44:26.137719 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 308/602] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:44:25 2022] POST /compute/v2.1/servers => generated 402 bytes in 806 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 11 14:44:26.153923 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-b47f3fdf-6882-49a4-beef-afda029335d2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:26.157961 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-b47f3fdf-6882-49a4-beef-afda029335d2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:26.170586 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-4172993f-7dab-486f-b0f1-70002c6f6683 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "DELETE /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 204 len: 0 microversion: 2.1 time: 0.152407 Jul 11 14:44:26.171378 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 295/603] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:44:26 2022] DELETE /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 11 14:44:26.183221 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e9ae53ad-b94a-42a8-9e1f-e29472d6b17f tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:26.183989 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-e9ae53ad-b94a-42a8-9e1f-e29472d6b17f tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:26.191194 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e9ae53ad-b94a-42a8-9e1f-e29472d6b17f tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:26.191640 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e9ae53ad-b94a-42a8-9e1f-e29472d6b17f tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:26.192404 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e9ae53ad-b94a-42a8-9e1f-e29472d6b17f tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:26.192834 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e9ae53ad-b94a-42a8-9e1f-e29472d6b17f tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:26.237533 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-b47f3fdf-6882-49a4-beef-afda029335d2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:26.247475 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-b47f3fdf-6882-49a4-beef-afda029335d2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 200 len: 1241 microversion: 2.1 time: 0.102284 Jul 11 14:44:26.248062 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 309/604] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:26 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 1241 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:44:26.338323 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e9ae53ad-b94a-42a8-9e1f-e29472d6b17f tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:26.345811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e9ae53ad-b94a-42a8-9e1f-e29472d6b17f tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1513 microversion: 2.1 time: 0.167904 Jul 11 14:44:26.346381 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 296/605] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:26 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1513 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:26.779702 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8881f3a3-fdb5-41fd-8c9d-38313cddb1ce tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:26.780258 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8881f3a3-fdb5-41fd-8c9d-38313cddb1ce tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:26.787187 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8881f3a3-fdb5-41fd-8c9d-38313cddb1ce tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:26.787643 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8881f3a3-fdb5-41fd-8c9d-38313cddb1ce tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:26.788417 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8881f3a3-fdb5-41fd-8c9d-38313cddb1ce tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:26.789463 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8881f3a3-fdb5-41fd-8c9d-38313cddb1ce tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:26.932466 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8881f3a3-fdb5-41fd-8c9d-38313cddb1ce tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:26.940451 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8881f3a3-fdb5-41fd-8c9d-38313cddb1ce tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.164757 Jul 11 14:44:26.941235 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 310/606] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:26 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:27.263747 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-44f4f13b-7baa-42af-8813-95bba26e5264 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:27.264347 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-44f4f13b-7baa-42af-8813-95bba26e5264 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:27.270869 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-44f4f13b-7baa-42af-8813-95bba26e5264 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:27.271311 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-44f4f13b-7baa-42af-8813-95bba26e5264 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:27.272059 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-44f4f13b-7baa-42af-8813-95bba26e5264 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:27.272487 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-44f4f13b-7baa-42af-8813-95bba26e5264 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:27.336077 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-44f4f13b-7baa-42af-8813-95bba26e5264 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:27.343429 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-44f4f13b-7baa-42af-8813-95bba26e5264 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 200 len: 1245 microversion: 2.1 time: 0.083626 Jul 11 14:44:27.343985 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 297/607] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:27 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 1245 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:27.363514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e7ad5759-5d3f-423b-8591-d9c44e35919a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:27.364123 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e7ad5759-5d3f-423b-8591-d9c44e35919a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:27.370009 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e7ad5759-5d3f-423b-8591-d9c44e35919a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:27.370481 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e7ad5759-5d3f-423b-8591-d9c44e35919a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:27.371229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e7ad5759-5d3f-423b-8591-d9c44e35919a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:27.371660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e7ad5759-5d3f-423b-8591-d9c44e35919a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:27.498861 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e7ad5759-5d3f-423b-8591-d9c44e35919a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:27.506221 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e7ad5759-5d3f-423b-8591-d9c44e35919a tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1513 microversion: 2.1 time: 0.146516 Jul 11 14:44:27.506809 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 311/608] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:27 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1513 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:27.958085 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c7cfc033-3d4a-4b61-9e02-c1c29cf4e9b9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:27.958648 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c7cfc033-3d4a-4b61-9e02-c1c29cf4e9b9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:27.964682 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c7cfc033-3d4a-4b61-9e02-c1c29cf4e9b9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:27.965138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c7cfc033-3d4a-4b61-9e02-c1c29cf4e9b9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:27.965898 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c7cfc033-3d4a-4b61-9e02-c1c29cf4e9b9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:27.966336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c7cfc033-3d4a-4b61-9e02-c1c29cf4e9b9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:28.105380 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c7cfc033-3d4a-4b61-9e02-c1c29cf4e9b9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:28.113501 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c7cfc033-3d4a-4b61-9e02-c1c29cf4e9b9 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.159658 Jul 11 14:44:28.114091 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 298/609] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:27 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:28.359479 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9b7354a8-51be-4614-b204-ca877eca51fb tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:28.360179 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9b7354a8-51be-4614-b204-ca877eca51fb tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:28.367473 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9b7354a8-51be-4614-b204-ca877eca51fb tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:28.367915 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9b7354a8-51be-4614-b204-ca877eca51fb tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:28.368677 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9b7354a8-51be-4614-b204-ca877eca51fb tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:28.369106 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9b7354a8-51be-4614-b204-ca877eca51fb tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:28.432608 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9b7354a8-51be-4614-b204-ca877eca51fb tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:28.440031 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9b7354a8-51be-4614-b204-ca877eca51fb tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 200 len: 1293 microversion: 2.1 time: 0.084589 Jul 11 14:44:28.440593 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 312/610] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:28 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 1293 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:28.523721 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f3558240-79cc-41ff-a49c-9c6f6f1cf414 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:28.524326 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f3558240-79cc-41ff-a49c-9c6f6f1cf414 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:28.530767 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f3558240-79cc-41ff-a49c-9c6f6f1cf414 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:28.531212 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f3558240-79cc-41ff-a49c-9c6f6f1cf414 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:28.531951 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f3558240-79cc-41ff-a49c-9c6f6f1cf414 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:28.532379 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f3558240-79cc-41ff-a49c-9c6f6f1cf414 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:28.547187 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=4,nova_cell1:INSERT=4 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:44:28.662163 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f3558240-79cc-41ff-a49c-9c6f6f1cf414 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:28.670396 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f3558240-79cc-41ff-a49c-9c6f6f1cf414 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1513 microversion: 2.1 time: 0.150526 Jul 11 14:44:28.671001 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 299/611] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:28 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1513 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:29.131697 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-7b804954-0265-4129-82a3-70c80b20235a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:29.133593 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-7b804954-0265-4129-82a3-70c80b20235a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:29.140847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7b804954-0265-4129-82a3-70c80b20235a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:29.141465 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7b804954-0265-4129-82a3-70c80b20235a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:29.142786 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7b804954-0265-4129-82a3-70c80b20235a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:29.143470 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7b804954-0265-4129-82a3-70c80b20235a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:29.351394 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7b804954-0265-4129-82a3-70c80b20235a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:29.359894 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-7b804954-0265-4129-82a3-70c80b20235a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.232163 Jul 11 14:44:29.360903 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 313/612] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:29 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:29.458220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-cbf92356-ade1-48b5-875c-d97452974796 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:29.458869 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-cbf92356-ade1-48b5-875c-d97452974796 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:29.465266 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cbf92356-ade1-48b5-875c-d97452974796 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:29.465705 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cbf92356-ade1-48b5-875c-d97452974796 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:29.466492 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cbf92356-ade1-48b5-875c-d97452974796 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:29.466927 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cbf92356-ade1-48b5-875c-d97452974796 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:29.512790 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-e1c8907d-3985-419b-aa57-940f08d9cc56 req-66f3c596-b543-4404-9bba-96278b772915 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7e398a3c-0f26-422e-8cb2-828013595327", "name": "network-vif-deleted", "tag": "1e493ea0-ba58-44b8-906b-71b97862dd76"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:44:29.522224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-e1c8907d-3985-419b-aa57-940f08d9cc56 req-66f3c596-b543-4404-9bba-96278b772915 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:29.522832 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-e1c8907d-3985-419b-aa57-940f08d9cc56 req-66f3c596-b543-4404-9bba-96278b772915 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:29.535317 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-e1c8907d-3985-419b-aa57-940f08d9cc56 req-66f3c596-b543-4404-9bba-96278b772915 service nova] Creating event network-vif-deleted:1e493ea0-ba58-44b8-906b-71b97862dd76 for instance 7e398a3c-0f26-422e-8cb2-828013595327 on hv-cinder-maste Jul 11 14:44:29.543385 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-e1c8907d-3985-419b-aa57-940f08d9cc56 req-66f3c596-b543-4404-9bba-96278b772915 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034081 Jul 11 14:44:29.544292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 314/613] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:44:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:44:29.545561 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-cbf92356-ade1-48b5-875c-d97452974796 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:29.556048 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-cbf92356-ade1-48b5-875c-d97452974796 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 200 len: 1299 microversion: 2.1 time: 0.102381 Jul 11 14:44:29.556627 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 300/614] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:29 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 1299 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:29.699573 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-38653015-79c6-49ee-ae9d-2b3e81d79169 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:29.700370 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-38653015-79c6-49ee-ae9d-2b3e81d79169 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:29.707136 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-38653015-79c6-49ee-ae9d-2b3e81d79169 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:29.707595 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-38653015-79c6-49ee-ae9d-2b3e81d79169 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:29.708392 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-38653015-79c6-49ee-ae9d-2b3e81d79169 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:29.708831 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-38653015-79c6-49ee-ae9d-2b3e81d79169 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:29.741520 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-38653015-79c6-49ee-ae9d-2b3e81d79169 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:29.741975 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-38653015-79c6-49ee-ae9d-2b3e81d79169 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:29.742460 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.objects.instance [None req-38653015-79c6-49ee-ae9d-2b3e81d79169 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lazy-loading 'fault' on Instance uuid 7e398a3c-0f26-422e-8cb2-828013595327 {{(pid=130159) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:44:29.784854 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-38653015-79c6-49ee-ae9d-2b3e81d79169 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b7457a38a0594e8da4c053a5c6a4aa15', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '8e0ccc1e4683478889d90814a1305507', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:29.788355 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:44:29.793991 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-38653015-79c6-49ee-ae9d-2b3e81d79169 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 200 len: 1308 microversion: 2.1 time: 0.109241 Jul 11 14:44:29.794594 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 315/615] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:29 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 1308 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:30.377414 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2023e2ce-bcc1-49eb-979b-344ec3d64410 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:30.378056 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2023e2ce-bcc1-49eb-979b-344ec3d64410 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:30.385214 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2023e2ce-bcc1-49eb-979b-344ec3d64410 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:30.385666 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2023e2ce-bcc1-49eb-979b-344ec3d64410 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:30.386458 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2023e2ce-bcc1-49eb-979b-344ec3d64410 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:30.386895 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2023e2ce-bcc1-49eb-979b-344ec3d64410 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:30.575852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-90561588-46fe-4371-b4cb-7bd6730732dd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:30.577358 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-90561588-46fe-4371-b4cb-7bd6730732dd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:30.589766 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-90561588-46fe-4371-b4cb-7bd6730732dd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:30.590299 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-90561588-46fe-4371-b4cb-7bd6730732dd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:30.591456 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-90561588-46fe-4371-b4cb-7bd6730732dd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:30.592050 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-90561588-46fe-4371-b4cb-7bd6730732dd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:30.609832 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2023e2ce-bcc1-49eb-979b-344ec3d64410 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:30.622182 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2023e2ce-bcc1-49eb-979b-344ec3d64410 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.248446 Jul 11 14:44:30.623242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 301/616] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:30 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:30.756194 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-90561588-46fe-4371-b4cb-7bd6730732dd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:30.765224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-90561588-46fe-4371-b4cb-7bd6730732dd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 200 len: 1367 microversion: 2.1 time: 0.195410 Jul 11 14:44:30.766039 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 316/617] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:30 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 1367 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:30.816253 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-39ea1d15-4b6a-4cff-9fd3-a351a7ec7ab6 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:30.817854 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-39ea1d15-4b6a-4cff-9fd3-a351a7ec7ab6 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] [instance: 7e398a3c-0f26-422e-8cb2-828013595327] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:30.820071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-5c02af37-3f1b-4695-93e2-e58e5dad9974 req-fd9add59-5a0e-4533-9389-30b933cc7665 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1f161996-6f14-46a9-9200-c8763cb67e21", "tag": "2ea3ae8a-9690-4d6d-82ab-c4549ae55695"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:44:30.827844 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-39ea1d15-4b6a-4cff-9fd3-a351a7ec7ab6 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:30.828537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-39ea1d15-4b6a-4cff-9fd3-a351a7ec7ab6 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:30.829675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-39ea1d15-4b6a-4cff-9fd3-a351a7ec7ab6 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:30.830296 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-39ea1d15-4b6a-4cff-9fd3-a351a7ec7ab6 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:30.833861 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-5c02af37-3f1b-4695-93e2-e58e5dad9974 req-fd9add59-5a0e-4533-9389-30b933cc7665 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:30.834419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-5c02af37-3f1b-4695-93e2-e58e5dad9974 req-fd9add59-5a0e-4533-9389-30b933cc7665 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:30.854717 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-39ea1d15-4b6a-4cff-9fd3-a351a7ec7ab6 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] HTTP exception thrown: Instance 7e398a3c-0f26-422e-8cb2-828013595327 could not be found. Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-39ea1d15-4b6a-4cff-9fd3-a351a7ec7ab6 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Request method failure captured: Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 HTTP/1.1 Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDa2yMolE6SY6wlljmuTPJUnmChUOjyVv0PLygzOqJokF5mbLB0jheLz-pwfVGXYRs-wnAtODBfHJ5JlZwKiBzLjmpfOWcF5q6ujET7O5KmqIrZUAvGgM5IIBAgguVRVxupkMcG6kVw5ULLQzzuzLD73kI6zWOiYZg070bqMburZ8yM Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: 8e0ccc1e4683478889d90814a1305507 Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1565526209 Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:44:30.856905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:44:30.858361 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/8e0ccc1e4683478889d90814a1305507"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/8e0ccc1e4683478889d90814a1305507"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_8e0ccc1e4683478889d90814a1305507"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/8e0ccc1e4683478889d90814a1305507"}]}] Jul 11 14:44:30.858361 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-VolumesSnapshotTestJSON-1565526209 Jul 11 14:44:30.858361 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: 8e0ccc1e4683478889d90814a1305507 Jul 11 14:44:30.858361 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1565526209 Jul 11 14:44:30.858361 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-VolumesSnapshotTestJSON-1565526209-project Jul 11 14:44:30.858361 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:44:30.858361 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:44:30.858361 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: b7457a38a0594e8da4c053a5c6a4aa15 Jul 11 14:44:30.858361 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-VolumesSnapshotTestJSON-1565526209-project Jul 11 14:44:30.858361 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:44:30.858361 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance 7e398a3c-0f26-422e-8cb2-828013595327 could not be found. Jul 11 14:44:30.858361 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '7e398a3c-0f26-422e-8cb2-828013595327'} Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7e398a3c-0f26-422e-8cb2-828013595327 could not be found. Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:44:30.859236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:44:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:44:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:44:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:44:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:44:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:44:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:44:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:44:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7e398a3c-0f26-422e-8cb2-828013595327 could not be found. Jul 11 14:44:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:44:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:44:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:44:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:44:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:44:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:44:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:44:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:44:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7e398a3c-0f26-422e-8cb2-828013595327 could not be found. Jul 11 14:44:30.860516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:44:30.861820 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-5c02af37-3f1b-4695-93e2-e58e5dad9974 req-fd9add59-5a0e-4533-9389-30b933cc7665 service nova] Creating event network-changed:2ea3ae8a-9690-4d6d-82ab-c4549ae55695 for instance 1f161996-6f14-46a9-9200-c8763cb67e21 on hv-cinder-maste Jul 11 14:44:30.863564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-39ea1d15-4b6a-4cff-9fd3-a351a7ec7ab6 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] Returning 404 to user: Instance 7e398a3c-0f26-422e-8cb2-828013595327 could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:44:30.864845 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-39ea1d15-4b6a-4cff-9fd3-a351a7ec7ab6 tempest-VolumesSnapshotTestJSON-1565526209 tempest-VolumesSnapshotTestJSON-1565526209-project] 10.222.0.72 "GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327" status: 404 len: 111 microversion: 2.1 time: 0.055610 Jul 11 14:44:30.868929 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-5c02af37-3f1b-4695-93e2-e58e5dad9974 req-fd9add59-5a0e-4533-9389-30b933cc7665 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060958 Jul 11 14:44:30.869320 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 317/618] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:30 2022] GET /compute/v2.1/servers/7e398a3c-0f26-422e-8cb2-828013595327 => generated 111 bytes in 62 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:44:30.876875 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 302/619] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:44:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:31.641275 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-58cfce52-1a75-4237-a3dd-e842cc4b00be tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:31.641882 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-58cfce52-1a75-4237-a3dd-e842cc4b00be tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:31.648919 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-58cfce52-1a75-4237-a3dd-e842cc4b00be tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:31.649502 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-58cfce52-1a75-4237-a3dd-e842cc4b00be tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:31.650358 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-58cfce52-1a75-4237-a3dd-e842cc4b00be tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:31.651048 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-58cfce52-1a75-4237-a3dd-e842cc4b00be tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:31.781590 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-58cfce52-1a75-4237-a3dd-e842cc4b00be tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:31.783167 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-459e9b12-d983-4ff7-bcfd-e63c25f57c79 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:31.783873 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-459e9b12-d983-4ff7-bcfd-e63c25f57c79 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:31.790628 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-459e9b12-d983-4ff7-bcfd-e63c25f57c79 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:31.790823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-58cfce52-1a75-4237-a3dd-e842cc4b00be tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.153169 Jul 11 14:44:31.791073 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-459e9b12-d983-4ff7-bcfd-e63c25f57c79 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:31.791391 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 318/620] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:31 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:31.791909 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-459e9b12-d983-4ff7-bcfd-e63c25f57c79 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:31.792457 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-459e9b12-d983-4ff7-bcfd-e63c25f57c79 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:31.925737 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-459e9b12-d983-4ff7-bcfd-e63c25f57c79 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:31.933685 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-459e9b12-d983-4ff7-bcfd-e63c25f57c79 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 200 len: 1367 microversion: 2.1 time: 0.154598 Jul 11 14:44:31.934250 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 303/621] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:31 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:44:32.809514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c171dfdd-60b9-4203-9661-f2ee1ec1b0ba tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:32.810222 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c171dfdd-60b9-4203-9661-f2ee1ec1b0ba tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:32.812695 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:44:32.818120 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c171dfdd-60b9-4203-9661-f2ee1ec1b0ba tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:32.818627 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c171dfdd-60b9-4203-9661-f2ee1ec1b0ba tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:32.819437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c171dfdd-60b9-4203-9661-f2ee1ec1b0ba tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:32.819874 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c171dfdd-60b9-4203-9661-f2ee1ec1b0ba tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:32.950898 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c3e2802d-036e-4f82-a1fb-9ef2f848a234 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:32.951648 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c3e2802d-036e-4f82-a1fb-9ef2f848a234 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:32.954190 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c171dfdd-60b9-4203-9661-f2ee1ec1b0ba tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:32.959061 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c3e2802d-036e-4f82-a1fb-9ef2f848a234 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:32.959567 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c3e2802d-036e-4f82-a1fb-9ef2f848a234 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:32.960399 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c3e2802d-036e-4f82-a1fb-9ef2f848a234 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:32.960833 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c3e2802d-036e-4f82-a1fb-9ef2f848a234 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:32.962547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c171dfdd-60b9-4203-9661-f2ee1ec1b0ba tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.157105 Jul 11 14:44:32.963121 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 319/622] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:32 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:33.310609 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c3e2802d-036e-4f82-a1fb-9ef2f848a234 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:33.317972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c3e2802d-036e-4f82-a1fb-9ef2f848a234 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 200 len: 1367 microversion: 2.1 time: 0.371025 Jul 11 14:44:33.318568 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 304/623] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:32 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 1367 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:44:33.979929 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8b69a9b1-f570-4a17-895c-d4c9d89545e1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:33.980474 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8b69a9b1-f570-4a17-895c-d4c9d89545e1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:33.986907 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8b69a9b1-f570-4a17-895c-d4c9d89545e1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:33.987348 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8b69a9b1-f570-4a17-895c-d4c9d89545e1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:33.988095 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8b69a9b1-f570-4a17-895c-d4c9d89545e1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:33.988529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8b69a9b1-f570-4a17-895c-d4c9d89545e1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:34.116807 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8b69a9b1-f570-4a17-895c-d4c9d89545e1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:34.126035 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8b69a9b1-f570-4a17-895c-d4c9d89545e1 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.150015 Jul 11 14:44:34.126674 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 320/624] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:33 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:34.335337 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-eddb990f-b8fb-4b43-b05c-dc9b2d24de0b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:34.335949 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-eddb990f-b8fb-4b43-b05c-dc9b2d24de0b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:34.338396 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=4 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:44:34.343854 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eddb990f-b8fb-4b43-b05c-dc9b2d24de0b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:34.344300 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eddb990f-b8fb-4b43-b05c-dc9b2d24de0b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:34.345051 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eddb990f-b8fb-4b43-b05c-dc9b2d24de0b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:34.345484 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eddb990f-b8fb-4b43-b05c-dc9b2d24de0b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:34.473199 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-eddb990f-b8fb-4b43-b05c-dc9b2d24de0b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:34.480659 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-eddb990f-b8fb-4b43-b05c-dc9b2d24de0b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 200 len: 1367 microversion: 2.1 time: 0.149407 Jul 11 14:44:34.481236 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 305/625] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:34 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 1367 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:35.143777 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c9d23448-9361-482e-9e8f-186ab30b1a1b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:35.144384 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c9d23448-9361-482e-9e8f-186ab30b1a1b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:35.151040 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c9d23448-9361-482e-9e8f-186ab30b1a1b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:35.151500 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c9d23448-9361-482e-9e8f-186ab30b1a1b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:35.152278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c9d23448-9361-482e-9e8f-186ab30b1a1b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:35.152719 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c9d23448-9361-482e-9e8f-186ab30b1a1b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:35.283214 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c9d23448-9361-482e-9e8f-186ab30b1a1b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:35.291788 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c9d23448-9361-482e-9e8f-186ab30b1a1b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.151795 Jul 11 14:44:35.292392 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 321/626] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:35 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:35.498641 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-634e22d4-6dbb-4aa7-9534-81048ffc1769 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:35.499749 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-634e22d4-6dbb-4aa7-9534-81048ffc1769 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:35.505687 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-634e22d4-6dbb-4aa7-9534-81048ffc1769 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:35.506131 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-634e22d4-6dbb-4aa7-9534-81048ffc1769 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:35.508306 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-634e22d4-6dbb-4aa7-9534-81048ffc1769 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:35.508841 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-634e22d4-6dbb-4aa7-9534-81048ffc1769 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:35.645854 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-634e22d4-6dbb-4aa7-9534-81048ffc1769 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:35.653417 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-634e22d4-6dbb-4aa7-9534-81048ffc1769 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 200 len: 1367 microversion: 2.1 time: 0.158718 Jul 11 14:44:35.653920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 306/627] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:35 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 1367 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:36.309118 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-24de2855-740a-4b7c-a882-9c97c7de4f4d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:36.309738 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-24de2855-740a-4b7c-a882-9c97c7de4f4d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:36.311678 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=2 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:44:36.312833 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=8 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:44:36.317790 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-24de2855-740a-4b7c-a882-9c97c7de4f4d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:36.318240 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-24de2855-740a-4b7c-a882-9c97c7de4f4d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:36.319040 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-24de2855-740a-4b7c-a882-9c97c7de4f4d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:36.319970 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-24de2855-740a-4b7c-a882-9c97c7de4f4d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:36.453781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-24de2855-740a-4b7c-a882-9c97c7de4f4d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:36.462019 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-24de2855-740a-4b7c-a882-9c97c7de4f4d tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.156930 Jul 11 14:44:36.462617 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 322/628] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:36 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:36.670986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-71873d13-a35f-47dd-adc9-9e4392f2ea59 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:36.671876 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-71873d13-a35f-47dd-adc9-9e4392f2ea59 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:36.679544 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-71873d13-a35f-47dd-adc9-9e4392f2ea59 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:36.680068 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-71873d13-a35f-47dd-adc9-9e4392f2ea59 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:36.681020 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-71873d13-a35f-47dd-adc9-9e4392f2ea59 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:36.681683 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-71873d13-a35f-47dd-adc9-9e4392f2ea59 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:36.842189 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-71873d13-a35f-47dd-adc9-9e4392f2ea59 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:36.852112 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-71873d13-a35f-47dd-adc9-9e4392f2ea59 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 200 len: 1367 microversion: 2.1 time: 0.185719 Jul 11 14:44:36.854958 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 307/629] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:36 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 1367 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:37.479963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-21427cb7-8d7c-4ad0-8e8f-b2a469fb26ac tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:37.480815 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-21427cb7-8d7c-4ad0-8e8f-b2a469fb26ac tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:37.487532 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-21427cb7-8d7c-4ad0-8e8f-b2a469fb26ac tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:37.487987 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-21427cb7-8d7c-4ad0-8e8f-b2a469fb26ac tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:37.488796 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-21427cb7-8d7c-4ad0-8e8f-b2a469fb26ac tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:37.489266 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-21427cb7-8d7c-4ad0-8e8f-b2a469fb26ac tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:37.625353 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-21427cb7-8d7c-4ad0-8e8f-b2a469fb26ac tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:37.633670 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-21427cb7-8d7c-4ad0-8e8f-b2a469fb26ac tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.158140 Jul 11 14:44:37.634250 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 323/630] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:37 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:37.678416 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-6e3a36dc-cfc3-48de-8e32-6ae106a9b28e req-af9f3eb5-d1df-4a94-9c98-0067e4e67688 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1f161996-6f14-46a9-9200-c8763cb67e21", "name": "network-vif-plugged", "status": "completed", "tag": "2ea3ae8a-9690-4d6d-82ab-c4549ae55695"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:44:37.688067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-6e3a36dc-cfc3-48de-8e32-6ae106a9b28e req-af9f3eb5-d1df-4a94-9c98-0067e4e67688 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:37.689085 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-6e3a36dc-cfc3-48de-8e32-6ae106a9b28e req-af9f3eb5-d1df-4a94-9c98-0067e4e67688 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:37.700270 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-6e3a36dc-cfc3-48de-8e32-6ae106a9b28e req-af9f3eb5-d1df-4a94-9c98-0067e4e67688 service nova] Creating event network-vif-plugged:2ea3ae8a-9690-4d6d-82ab-c4549ae55695 for instance 1f161996-6f14-46a9-9200-c8763cb67e21 on hv-cinder-maste Jul 11 14:44:37.705078 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-6e3a36dc-cfc3-48de-8e32-6ae106a9b28e req-af9f3eb5-d1df-4a94-9c98-0067e4e67688 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030538 Jul 11 14:44:37.705661 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 308/631] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:44:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:37.870342 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-50927f31-cbe1-40cb-a551-39cab06b1f98 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:37.871256 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-50927f31-cbe1-40cb-a551-39cab06b1f98 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:37.877364 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-50927f31-cbe1-40cb-a551-39cab06b1f98 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:37.877922 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-50927f31-cbe1-40cb-a551-39cab06b1f98 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:37.878676 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-50927f31-cbe1-40cb-a551-39cab06b1f98 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:37.879122 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-50927f31-cbe1-40cb-a551-39cab06b1f98 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:38.017012 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-50927f31-cbe1-40cb-a551-39cab06b1f98 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:38.025266 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-50927f31-cbe1-40cb-a551-39cab06b1f98 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 200 len: 1367 microversion: 2.1 time: 0.159826 Jul 11 14:44:38.026201 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 324/632] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:37 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 1367 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:38.651223 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8d4464b0-3b8f-48de-8fa1-ad971eed2b0e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:38.651827 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-8d4464b0-3b8f-48de-8fa1-ad971eed2b0e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:38.658117 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d4464b0-3b8f-48de-8fa1-ad971eed2b0e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:38.658593 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d4464b0-3b8f-48de-8fa1-ad971eed2b0e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:38.659348 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d4464b0-3b8f-48de-8fa1-ad971eed2b0e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:38.659776 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8d4464b0-3b8f-48de-8fa1-ad971eed2b0e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:38.794010 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8d4464b0-3b8f-48de-8fa1-ad971eed2b0e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:38.802024 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8d4464b0-3b8f-48de-8fa1-ad971eed2b0e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.155082 Jul 11 14:44:38.802609 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 309/633] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:38 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:39.042738 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-6d1bcbcd-a9af-4874-a417-020161556d26 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:39.043345 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-6d1bcbcd-a9af-4874-a417-020161556d26 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:39.049807 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6d1bcbcd-a9af-4874-a417-020161556d26 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:39.050250 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6d1bcbcd-a9af-4874-a417-020161556d26 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:39.051035 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6d1bcbcd-a9af-4874-a417-020161556d26 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:39.051466 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6d1bcbcd-a9af-4874-a417-020161556d26 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:39.185037 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-6d1bcbcd-a9af-4874-a417-020161556d26 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:39.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-6d1bcbcd-a9af-4874-a417-020161556d26 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 200 len: 1367 microversion: 2.1 time: 0.154775 Jul 11 14:44:39.193725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 325/634] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:39 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 1367 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:39.819091 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-014b92d0-879f-4b06-b9d1-a7caf6872b05 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:39.820252 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-014b92d0-879f-4b06-b9d1-a7caf6872b05 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:39.826835 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-014b92d0-879f-4b06-b9d1-a7caf6872b05 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:39.827285 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-014b92d0-879f-4b06-b9d1-a7caf6872b05 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:39.828040 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-014b92d0-879f-4b06-b9d1-a7caf6872b05 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:39.828461 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-014b92d0-879f-4b06-b9d1-a7caf6872b05 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:39.958981 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-014b92d0-879f-4b06-b9d1-a7caf6872b05 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:39.967418 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-014b92d0-879f-4b06-b9d1-a7caf6872b05 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.152257 Jul 11 14:44:39.967993 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 310/635] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:39 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:40.211322 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a18900a2-bcdc-499d-864a-b7e9f431ed70 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:40.211877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a18900a2-bcdc-499d-864a-b7e9f431ed70 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:40.218647 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a18900a2-bcdc-499d-864a-b7e9f431ed70 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:40.219106 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a18900a2-bcdc-499d-864a-b7e9f431ed70 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:40.219877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a18900a2-bcdc-499d-864a-b7e9f431ed70 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:40.220322 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a18900a2-bcdc-499d-864a-b7e9f431ed70 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:40.356911 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a18900a2-bcdc-499d-864a-b7e9f431ed70 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:40.364406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a18900a2-bcdc-499d-864a-b7e9f431ed70 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 200 len: 1557 microversion: 2.1 time: 0.157846 Jul 11 14:44:40.365032 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 326/636] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:40 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 1557 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:40.382085 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-1bda33c7-90db-419f-9214-9e5b551f3884 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:40.382672 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-1bda33c7-90db-419f-9214-9e5b551f3884 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:40.389035 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1bda33c7-90db-419f-9214-9e5b551f3884 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:40.389481 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1bda33c7-90db-419f-9214-9e5b551f3884 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:40.390247 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1bda33c7-90db-419f-9214-9e5b551f3884 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:40.390718 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1bda33c7-90db-419f-9214-9e5b551f3884 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:40.477669 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-1bda33c7-90db-419f-9214-9e5b551f3884 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.099576 Jul 11 14:44:40.478272 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 311/637] 10.222.0.72 () {58 vars in 1274 bytes} [Mon Jul 11 14:44:40 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-interface => generated 285 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:40.985057 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-63257776-d640-4929-b911-54348baf33df tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:40.985662 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-63257776-d640-4929-b911-54348baf33df tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:40.992587 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-63257776-d640-4929-b911-54348baf33df tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:40.993030 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-63257776-d640-4929-b911-54348baf33df tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:40.993780 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-63257776-d640-4929-b911-54348baf33df tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:40.994218 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-63257776-d640-4929-b911-54348baf33df tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:41.130059 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-63257776-d640-4929-b911-54348baf33df tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:41.139352 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-63257776-d640-4929-b911-54348baf33df tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.158040 Jul 11 14:44:41.139952 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 327/638] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:40 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:41.457068 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-c5d2a3de-c1a5-4b4d-9985-149cd6ace08d req-c665e167-97b9-4603-9a27-72c321b5f787 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1f161996-6f14-46a9-9200-c8763cb67e21", "tag": "2ea3ae8a-9690-4d6d-82ab-c4549ae55695"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:44:41.461212 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=83,nova_api:DELETE=1,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:44:41.467139 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-c5d2a3de-c1a5-4b4d-9985-149cd6ace08d req-c665e167-97b9-4603-9a27-72c321b5f787 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:41.467589 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-c5d2a3de-c1a5-4b4d-9985-149cd6ace08d req-c665e167-97b9-4603-9a27-72c321b5f787 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:41.480774 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-c5d2a3de-c1a5-4b4d-9985-149cd6ace08d req-c665e167-97b9-4603-9a27-72c321b5f787 service nova] Creating event network-changed:2ea3ae8a-9690-4d6d-82ab-c4549ae55695 for instance 1f161996-6f14-46a9-9200-c8763cb67e21 on hv-cinder-maste Jul 11 14:44:41.487334 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-c5d2a3de-c1a5-4b4d-9985-149cd6ace08d req-c665e167-97b9-4603-9a27-72c321b5f787 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034517 Jul 11 14:44:41.487952 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 312/639] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:44:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:42.156622 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3ddd491f-0a5e-440e-be51-7626996e8952 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:42.157244 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3ddd491f-0a5e-440e-be51-7626996e8952 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:42.163623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3ddd491f-0a5e-440e-be51-7626996e8952 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:42.164282 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3ddd491f-0a5e-440e-be51-7626996e8952 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:42.165089 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3ddd491f-0a5e-440e-be51-7626996e8952 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:42.165534 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3ddd491f-0a5e-440e-be51-7626996e8952 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:42.306805 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3ddd491f-0a5e-440e-be51-7626996e8952 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:42.315879 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3ddd491f-0a5e-440e-be51-7626996e8952 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.163202 Jul 11 14:44:42.316463 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 328/640] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:42 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:43.335574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f8486acb-284e-4d3c-a897-29a1a5e90d3a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:43.336280 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f8486acb-284e-4d3c-a897-29a1a5e90d3a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:43.343523 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f8486acb-284e-4d3c-a897-29a1a5e90d3a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:43.343976 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f8486acb-284e-4d3c-a897-29a1a5e90d3a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:43.344796 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f8486acb-284e-4d3c-a897-29a1a5e90d3a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:43.345278 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f8486acb-284e-4d3c-a897-29a1a5e90d3a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:43.491851 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f8486acb-284e-4d3c-a897-29a1a5e90d3a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:43.501057 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f8486acb-284e-4d3c-a897-29a1a5e90d3a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.170976 Jul 11 14:44:43.501650 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 313/641] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:43 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:44.518259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-004fc529-a219-4408-b6d2-e1fd41398c89 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:44.518889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-004fc529-a219-4408-b6d2-e1fd41398c89 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:44.525824 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-004fc529-a219-4408-b6d2-e1fd41398c89 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:44.526268 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-004fc529-a219-4408-b6d2-e1fd41398c89 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:44.527053 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-004fc529-a219-4408-b6d2-e1fd41398c89 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:44.527490 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-004fc529-a219-4408-b6d2-e1fd41398c89 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:44.659433 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-004fc529-a219-4408-b6d2-e1fd41398c89 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:44.668023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-004fc529-a219-4408-b6d2-e1fd41398c89 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.153627 Jul 11 14:44:44.668595 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 329/642] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:44 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:45.685169 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a99e5a33-8cc8-40ab-a961-60a1fa7d8063 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:45.685785 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a99e5a33-8cc8-40ab-a961-60a1fa7d8063 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:45.692403 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a99e5a33-8cc8-40ab-a961-60a1fa7d8063 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:45.692856 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a99e5a33-8cc8-40ab-a961-60a1fa7d8063 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:45.693618 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a99e5a33-8cc8-40ab-a961-60a1fa7d8063 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:45.694081 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a99e5a33-8cc8-40ab-a961-60a1fa7d8063 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:45.825725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a99e5a33-8cc8-40ab-a961-60a1fa7d8063 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:45.834035 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a99e5a33-8cc8-40ab-a961-60a1fa7d8063 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.152828 Jul 11 14:44:45.834834 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 314/643] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:45 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:46.851782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-ed6ca5df-edc9-439d-a07e-0a1c98749ddb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:46.852369 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-ed6ca5df-edc9-439d-a07e-0a1c98749ddb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:46.859020 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ed6ca5df-edc9-439d-a07e-0a1c98749ddb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:46.859472 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ed6ca5df-edc9-439d-a07e-0a1c98749ddb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:46.860266 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ed6ca5df-edc9-439d-a07e-0a1c98749ddb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:46.860701 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ed6ca5df-edc9-439d-a07e-0a1c98749ddb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:47.001078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-ed6ca5df-edc9-439d-a07e-0a1c98749ddb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:47.009724 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-ed6ca5df-edc9-439d-a07e-0a1c98749ddb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1731 microversion: 2.1 time: 0.161980 Jul 11 14:44:47.010297 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 330/644] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:46 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1731 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:48.028122 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-b9c2e2c0-54b9-4a56-b34d-18f897de2a6b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:48.028958 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-b9c2e2c0-54b9-4a56-b34d-18f897de2a6b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:48.036654 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b9c2e2c0-54b9-4a56-b34d-18f897de2a6b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:48.037296 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b9c2e2c0-54b9-4a56-b34d-18f897de2a6b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:48.038465 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b9c2e2c0-54b9-4a56-b34d-18f897de2a6b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:48.039072 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b9c2e2c0-54b9-4a56-b34d-18f897de2a6b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:48.181151 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-b9c2e2c0-54b9-4a56-b34d-18f897de2a6b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:48.189678 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-b9c2e2c0-54b9-4a56-b34d-18f897de2a6b tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1711 microversion: 2.1 time: 0.166122 Jul 11 14:44:48.190288 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 315/645] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:48 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1711 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:48.312599 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-051edfaf-44a2-4570-a39b-3ba0eaec8b98 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:48.313174 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-051edfaf-44a2-4570-a39b-3ba0eaec8b98 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:48.320298 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-051edfaf-44a2-4570-a39b-3ba0eaec8b98 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:48.320751 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-051edfaf-44a2-4570-a39b-3ba0eaec8b98 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:48.321527 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-051edfaf-44a2-4570-a39b-3ba0eaec8b98 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:48.321971 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-051edfaf-44a2-4570-a39b-3ba0eaec8b98 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:48.407783 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-051edfaf-44a2-4570-a39b-3ba0eaec8b98 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] HTTP exception thrown: Cannot 'detach_volume' instance f66823cd-4fe8-42cf-ab7a-d7cd7afc289a while it is in vm_state rescued Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-051edfaf-44a2-4570-a39b-3ba0eaec8b98 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Request method failure captured: Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: DELETE /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-volume_attachments/2be6be9e-10da-4656-a426-f2a29458eb3c HTTP/1.1 Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZVO0vcEhdpPwwfJZnu1y3p7Tg0Q8GENrUVh0NohR9oAPNIjKFLY8v62KxMOiNHGfwF-EVGge1hPwlyTCX1uExRac1NToAISnaSDmVmpVWA-TKGS1rzgSZVkfiPxYTNCyZmEJBrMLK9NH_pMk0kTrm35o7KBc7uMvJqd_pTjsh0TKE Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: 98966d44ea88477b94215761cdc92ea7 Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1829929608 Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:44:48.409301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:44:48.410912 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/98966d44ea88477b94215761cdc92ea7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/98966d44ea88477b94215761cdc92ea7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_98966d44ea88477b94215761cdc92ea7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/98966d44ea88477b94215761cdc92ea7"}]}] Jul 11 14:44:48.410912 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1829929608 Jul 11 14:44:48.410912 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: 98966d44ea88477b94215761cdc92ea7 Jul 11 14:44:48.410912 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1829929608 Jul 11 14:44:48.410912 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-ServerRescueNegativeTestJSON-1829929608-project Jul 11 14:44:48.410912 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:44:48.410912 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:44:48.410912 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: fa478fb4b41e49c5b1f9291f925cca23 Jul 11 14:44:48.410912 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1829929608-project Jul 11 14:44:48.410912 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:44:48.410912 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Cannot 'detach_volume' instance f66823cd-4fe8-42cf-ab7a-d7cd7afc289a while it is in vm_state rescued Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'server_id': 'f66823cd-4fe8-42cf-ab7a-d7cd7afc289a', 'id': '2be6be9e-10da-4656-a426-f2a29458eb3c'} Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 145, in inner Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance f66823cd-4fe8-42cf-ab7a-d7cd7afc289a in vm_state rescued. Cannot detach_volume while the instance is in this state. Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:44:48.411729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:44:48.413561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Jul 11 14:44:48.413561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Jul 11 14:44:48.413561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Jul 11 14:44:48.413561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Jul 11 14:44:48.413561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance f66823cd-4fe8-42cf-ab7a-d7cd7afc289a while it is in vm_state rescued Jul 11 14:44:48.413561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:44:48.413561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:44:48.413561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:44:48.413561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:44:48.413561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:44:48.413561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:44:48.413561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:44:48.413561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:44:48.413561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance f66823cd-4fe8-42cf-ab7a-d7cd7afc289a while it is in vm_state rescued Jul 11 14:44:48.413561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:44:48.413561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-051edfaf-44a2-4570-a39b-3ba0eaec8b98 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Returning 409 to user: Cannot 'detach_volume' instance f66823cd-4fe8-42cf-ab7a-d7cd7afc289a while it is in vm_state rescued {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:44:48.414616 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-051edfaf-44a2-4570-a39b-3ba0eaec8b98 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "DELETE /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-volume_attachments/2be6be9e-10da-4656-a426-f2a29458eb3c" status: 409 len: 152 microversion: 2.1 time: 0.102255 Jul 11 14:44:48.414616 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 331/646] 10.222.0.72 () {58 vars in 1415 bytes} [Mon Jul 11 14:44:48 2022] DELETE /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-volume_attachments/2be6be9e-10da-4656-a426-f2a29458eb3c => generated 152 bytes in 103 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Jul 11 14:44:48.420782 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9042432c-8dd6-432b-9b42-e327e862cbc7 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:44:48.421333 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-9042432c-8dd6-432b-9b42-e327e862cbc7 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:48.427722 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9042432c-8dd6-432b-9b42-e327e862cbc7 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:48.428177 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9042432c-8dd6-432b-9b42-e327e862cbc7 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:48.428988 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9042432c-8dd6-432b-9b42-e327e862cbc7 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:48.429428 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9042432c-8dd6-432b-9b42-e327e862cbc7 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:48.510643 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9042432c-8dd6-432b-9b42-e327e862cbc7 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "POST /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/action" status: 202 len: 0 microversion: 2.1 time: 0.094192 Jul 11 14:44:48.511791 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 316/647] 10.222.0.72 () {60 vars in 1277 bytes} [Mon Jul 11 14:44:48 2022] POST /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/action => generated 0 bytes in 96 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:44:48.520690 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-7f7c0e8a-8543-4137-9409-b782c4e090f4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:48.521322 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-7f7c0e8a-8543-4137-9409-b782c4e090f4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:48.528775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7f7c0e8a-8543-4137-9409-b782c4e090f4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:48.529687 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7f7c0e8a-8543-4137-9409-b782c4e090f4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:48.530779 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7f7c0e8a-8543-4137-9409-b782c4e090f4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:48.531264 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7f7c0e8a-8543-4137-9409-b782c4e090f4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:48.723835 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7f7c0e8a-8543-4137-9409-b782c4e090f4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:48.734521 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-7f7c0e8a-8543-4137-9409-b782c4e090f4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1719 microversion: 2.1 time: 0.218348 Jul 11 14:44:48.735050 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 332/648] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:48 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1719 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:49.760893 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-477eee7e-4e6a-47e9-8ebc-1932e86009e8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:49.761625 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-477eee7e-4e6a-47e9-8ebc-1932e86009e8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:49.768392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-477eee7e-4e6a-47e9-8ebc-1932e86009e8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:49.768843 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-477eee7e-4e6a-47e9-8ebc-1932e86009e8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:49.769617 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-477eee7e-4e6a-47e9-8ebc-1932e86009e8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:49.770066 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-477eee7e-4e6a-47e9-8ebc-1932e86009e8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:49.785549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:44:49.921422 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-477eee7e-4e6a-47e9-8ebc-1932e86009e8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:49.930728 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-477eee7e-4e6a-47e9-8ebc-1932e86009e8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1719 microversion: 2.1 time: 0.173987 Jul 11 14:44:49.931314 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 317/649] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:49 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1719 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:50.948732 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2a9373f6-dd03-4478-b5e3-caf97b38b4bb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:50.949273 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2a9373f6-dd03-4478-b5e3-caf97b38b4bb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:50.956111 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2a9373f6-dd03-4478-b5e3-caf97b38b4bb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:50.956862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2a9373f6-dd03-4478-b5e3-caf97b38b4bb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:50.957635 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2a9373f6-dd03-4478-b5e3-caf97b38b4bb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:50.958079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2a9373f6-dd03-4478-b5e3-caf97b38b4bb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:50.962378 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=100 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:44:51.108732 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-2a9373f6-dd03-4478-b5e3-caf97b38b4bb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:51.117268 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2a9373f6-dd03-4478-b5e3-caf97b38b4bb tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1719 microversion: 2.1 time: 0.172414 Jul 11 14:44:51.117853 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 333/650] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:50 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1719 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:52.141134 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-65cc5153-fbe0-496a-8644-6069a2d599fa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:52.141764 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-65cc5153-fbe0-496a-8644-6069a2d599fa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:52.148280 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-65cc5153-fbe0-496a-8644-6069a2d599fa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:52.148731 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-65cc5153-fbe0-496a-8644-6069a2d599fa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:52.149494 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-65cc5153-fbe0-496a-8644-6069a2d599fa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:52.149940 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-65cc5153-fbe0-496a-8644-6069a2d599fa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:52.278079 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-65cc5153-fbe0-496a-8644-6069a2d599fa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:52.286521 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-65cc5153-fbe0-496a-8644-6069a2d599fa tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1725 microversion: 2.1 time: 0.149167 Jul 11 14:44:52.287100 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 318/651] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:44:52 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1725 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:52.304425 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a9356628-caa8-4ecd-b617-0368ed0522a2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:52.304984 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a9356628-caa8-4ecd-b617-0368ed0522a2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:52.311660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a9356628-caa8-4ecd-b617-0368ed0522a2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:52.312117 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a9356628-caa8-4ecd-b617-0368ed0522a2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:52.312888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a9356628-caa8-4ecd-b617-0368ed0522a2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:52.313333 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a9356628-caa8-4ecd-b617-0368ed0522a2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:52.393815 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a9356628-caa8-4ecd-b617-0368ed0522a2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.093787 Jul 11 14:44:52.394478 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 334/652] 10.222.0.72 () {58 vars in 1274 bytes} [Mon Jul 11 14:44:52 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-interface => generated 285 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:53.565818 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-1d6f2bd5-46da-42c4-8b49-959b76fe8ee2 req-0b08a2de-cefe-459e-b4af-fade305c43ff service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f66823cd-4fe8-42cf-ab7a-d7cd7afc289a", "tag": "5a30e035-3f54-418c-8ac6-380a40fc2f5d"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:44:53.576121 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-1d6f2bd5-46da-42c4-8b49-959b76fe8ee2 req-0b08a2de-cefe-459e-b4af-fade305c43ff service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:53.576672 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-1d6f2bd5-46da-42c4-8b49-959b76fe8ee2 req-0b08a2de-cefe-459e-b4af-fade305c43ff service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:53.589253 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-1d6f2bd5-46da-42c4-8b49-959b76fe8ee2 req-0b08a2de-cefe-459e-b4af-fade305c43ff service nova] Creating event network-changed:5a30e035-3f54-418c-8ac6-380a40fc2f5d for instance f66823cd-4fe8-42cf-ab7a-d7cd7afc289a on hv-cinder-maste Jul 11 14:44:53.595651 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-1d6f2bd5-46da-42c4-8b49-959b76fe8ee2 req-0b08a2de-cefe-459e-b4af-fade305c43ff service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034636 Jul 11 14:44:53.596289 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 319/653] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:44:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:55.003293 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-b08dc0b1-3f47-43de-97be-ba5a6b3aef7a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:55.003840 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-b08dc0b1-3f47-43de-97be-ba5a6b3aef7a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:55.009882 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b08dc0b1-3f47-43de-97be-ba5a6b3aef7a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:55.010330 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b08dc0b1-3f47-43de-97be-ba5a6b3aef7a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:55.011139 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b08dc0b1-3f47-43de-97be-ba5a6b3aef7a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:55.011577 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b08dc0b1-3f47-43de-97be-ba5a6b3aef7a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:55.042990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-b08dc0b1-3f47-43de-97be-ba5a6b3aef7a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/os-volume_attachments/ced20d51-419d-4832-a700-5c8fa50339ab" status: 200 len: 194 microversion: 2.20 time: 0.044186 Jul 11 14:44:55.043601 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 335/654] 10.222.0.72 () {60 vars in 1453 bytes} [Mon Jul 11 14:44:54 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/os-volume_attachments/ced20d51-419d-4832-a700-5c8fa50339ab => generated 194 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:44:55.119747 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3d739dfd-8160-4b0c-a174-fafdabb898de tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:55.120344 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-3d739dfd-8160-4b0c-a174-fafdabb898de tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:55.126891 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d739dfd-8160-4b0c-a174-fafdabb898de tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:55.127319 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d739dfd-8160-4b0c-a174-fafdabb898de tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:55.128121 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d739dfd-8160-4b0c-a174-fafdabb898de tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:55.128558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d739dfd-8160-4b0c-a174-fafdabb898de tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:55.236770 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.objects.instance [None req-3d739dfd-8160-4b0c-a174-fafdabb898de tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lazy-loading 'services' on Instance uuid 13da005a-e085-43aa-8a18-1fe85e4e95af {{(pid=130160) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:44:55.320688 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-3d739dfd-8160-4b0c-a174-fafdabb898de tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "DELETE /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/os-volume_attachments/ced20d51-419d-4832-a700-5c8fa50339ab" status: 202 len: 0 microversion: 2.20 time: 0.205866 Jul 11 14:44:55.321292 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 320/655] 10.222.0.72 () {60 vars in 1456 bytes} [Mon Jul 11 14:44:55 2022] DELETE /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/os-volume_attachments/ced20d51-419d-4832-a700-5c8fa50339ab => generated 0 bytes in 208 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 11 14:44:55.338902 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2919439d-53e1-42a1-ae98-07d1964474a0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:55.340052 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2919439d-53e1-42a1-ae98-07d1964474a0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:55.348189 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2919439d-53e1-42a1-ae98-07d1964474a0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:55.349296 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2919439d-53e1-42a1-ae98-07d1964474a0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:55.350790 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2919439d-53e1-42a1-ae98-07d1964474a0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:55.351304 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2919439d-53e1-42a1-ae98-07d1964474a0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:55.407761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2919439d-53e1-42a1-ae98-07d1964474a0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.073280 Jul 11 14:44:55.410491 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 336/656] 10.222.0.72 () {60 vars in 1342 bytes} [Mon Jul 11 14:44:55 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/os-volume_attachments => generated 197 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 11 14:44:55.613077 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-254e8948-eba3-44f5-9f4f-5c4d38e8958a req-bc911d7c-b9f7-47b1-868c-fa52d63ec856 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f66823cd-4fe8-42cf-ab7a-d7cd7afc289a", "tag": "5a30e035-3f54-418c-8ac6-380a40fc2f5d"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:44:55.623401 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-254e8948-eba3-44f5-9f4f-5c4d38e8958a req-bc911d7c-b9f7-47b1-868c-fa52d63ec856 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:55.623844 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-254e8948-eba3-44f5-9f4f-5c4d38e8958a req-bc911d7c-b9f7-47b1-868c-fa52d63ec856 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:55.635915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-254e8948-eba3-44f5-9f4f-5c4d38e8958a req-bc911d7c-b9f7-47b1-868c-fa52d63ec856 service nova] Creating event network-changed:5a30e035-3f54-418c-8ac6-380a40fc2f5d for instance f66823cd-4fe8-42cf-ab7a-d7cd7afc289a on hv-cinder-maste Jul 11 14:44:55.640621 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-254e8948-eba3-44f5-9f4f-5c4d38e8958a req-bc911d7c-b9f7-47b1-868c-fa52d63ec856 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033175 Jul 11 14:44:55.641214 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 321/657] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:44:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:44:56.423710 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-84166dab-710c-4590-94d1-f8b7df9595f4 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:56.424956 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-84166dab-710c-4590-94d1-f8b7df9595f4 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:56.432186 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84166dab-710c-4590-94d1-f8b7df9595f4 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:56.432639 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84166dab-710c-4590-94d1-f8b7df9595f4 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:56.433420 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84166dab-710c-4590-94d1-f8b7df9595f4 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:56.433859 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84166dab-710c-4590-94d1-f8b7df9595f4 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:56.467193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-84166dab-710c-4590-94d1-f8b7df9595f4 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047558 Jul 11 14:44:56.468095 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 337/658] 10.222.0.72 () {60 vars in 1342 bytes} [Mon Jul 11 14:44:56 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:44:57.485082 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3f14ba71-b53f-441c-bb30-f1a05e7fcb2c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:57.486875 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-3f14ba71-b53f-441c-bb30-f1a05e7fcb2c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:57.498181 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3f14ba71-b53f-441c-bb30-f1a05e7fcb2c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:57.499045 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3f14ba71-b53f-441c-bb30-f1a05e7fcb2c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:57.500441 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3f14ba71-b53f-441c-bb30-f1a05e7fcb2c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:57.501263 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3f14ba71-b53f-441c-bb30-f1a05e7fcb2c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:57.540024 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-3f14ba71-b53f-441c-bb30-f1a05e7fcb2c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.063822 Jul 11 14:44:57.540633 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 322/659] 10.222.0.72 () {60 vars in 1342 bytes} [Mon Jul 11 14:44:57 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af/os-volume_attachments => generated 25 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:44:57.601962 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-70f96f2c-47c4-456c-855a-120e2f8c6b0b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:57.602541 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-70f96f2c-47c4-456c-855a-120e2f8c6b0b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:57.609118 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-70f96f2c-47c4-456c-855a-120e2f8c6b0b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:57.609562 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-70f96f2c-47c4-456c-855a-120e2f8c6b0b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:57.610330 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-70f96f2c-47c4-456c-855a-120e2f8c6b0b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:57.610801 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-70f96f2c-47c4-456c-855a-120e2f8c6b0b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:57.637210 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-70f96f2c-47c4-456c-855a-120e2f8c6b0b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Going to try to terminate instance {{(pid=130159) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:44:57.716600 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-70f96f2c-47c4-456c-855a-120e2f8c6b0b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "DELETE /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 204 len: 0 microversion: 2.20 time: 0.118156 Jul 11 14:44:57.717729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 338/660] 10.222.0.72 () {60 vars in 1279 bytes} [Mon Jul 11 14:44:57 2022] DELETE /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 11 14:44:57.726422 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e45927c6-24cf-4996-ad57-91c20b04ce95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:57.727065 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-e45927c6-24cf-4996-ad57-91c20b04ce95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:57.733117 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e45927c6-24cf-4996-ad57-91c20b04ce95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:57.733806 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e45927c6-24cf-4996-ad57-91c20b04ce95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:57.734744 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e45927c6-24cf-4996-ad57-91c20b04ce95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:57.735465 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e45927c6-24cf-4996-ad57-91c20b04ce95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:57.878937 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e45927c6-24cf-4996-ad57-91c20b04ce95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:57.888199 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e45927c6-24cf-4996-ad57-91c20b04ce95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:57.889732 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e45927c6-24cf-4996-ad57-91c20b04ce95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:57.890956 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e45927c6-24cf-4996-ad57-91c20b04ce95 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.168525 Jul 11 14:44:57.891512 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 323/661] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:44:57 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:44:58.910139 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-ee79be80-f37e-4514-a0ba-986bede1778f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:44:58.910874 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-ee79be80-f37e-4514-a0ba-986bede1778f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:44:58.918205 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ee79be80-f37e-4514-a0ba-986bede1778f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:58.918689 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ee79be80-f37e-4514-a0ba-986bede1778f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:58.919449 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ee79be80-f37e-4514-a0ba-986bede1778f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:44:58.919879 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ee79be80-f37e-4514-a0ba-986bede1778f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:44:59.053000 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-ee79be80-f37e-4514-a0ba-986bede1778f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:59.063420 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-ee79be80-f37e-4514-a0ba-986bede1778f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:59.064939 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-ee79be80-f37e-4514-a0ba-986bede1778f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:44:59.066099 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-ee79be80-f37e-4514-a0ba-986bede1778f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.160294 Jul 11 14:44:59.066683 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 339/662] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:44:58 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:45:00.085717 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-4a02a738-e2a6-4329-853d-bacd97ddeb5b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:00.086490 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-4a02a738-e2a6-4329-853d-bacd97ddeb5b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:00.093281 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4a02a738-e2a6-4329-853d-bacd97ddeb5b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:00.093819 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4a02a738-e2a6-4329-853d-bacd97ddeb5b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:00.094713 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4a02a738-e2a6-4329-853d-bacd97ddeb5b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:00.095166 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4a02a738-e2a6-4329-853d-bacd97ddeb5b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:00.342233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4a02a738-e2a6-4329-853d-bacd97ddeb5b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:00.351034 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4a02a738-e2a6-4329-853d-bacd97ddeb5b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:00.352572 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4a02a738-e2a6-4329-853d-bacd97ddeb5b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:00.353761 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-4a02a738-e2a6-4329-853d-bacd97ddeb5b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 200 len: 1723 microversion: 2.20 time: 0.272871 Jul 11 14:45:00.354331 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 324/663] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:45:00 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 1723 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:45:00.679433 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-6505d411-8ca4-4bf0-a9b2-83a27da43b3b tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:00.680413 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-6505d411-8ca4-4bf0-a9b2-83a27da43b3b tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:00.687069 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6505d411-8ca4-4bf0-a9b2-83a27da43b3b tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:00.687519 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6505d411-8ca4-4bf0-a9b2-83a27da43b3b tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:00.688286 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6505d411-8ca4-4bf0-a9b2-83a27da43b3b tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:00.688833 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6505d411-8ca4-4bf0-a9b2-83a27da43b3b tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:00.770097 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-6505d411-8ca4-4bf0-a9b2-83a27da43b3b tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "POST /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/action" status: 202 len: 0 microversion: 2.1 time: 0.096526 Jul 11 14:45:00.770797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 340/664] 10.222.0.72 () {60 vars in 1277 bytes} [Mon Jul 11 14:45:00 2022] POST /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/action => generated 0 bytes in 98 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:45:00.779481 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d6a75477-a0ab-4334-a790-b288a837ac34 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:00.780097 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d6a75477-a0ab-4334-a790-b288a837ac34 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:00.791926 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d6a75477-a0ab-4334-a790-b288a837ac34 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:00.792383 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d6a75477-a0ab-4334-a790-b288a837ac34 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:00.793149 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d6a75477-a0ab-4334-a790-b288a837ac34 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:00.793640 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d6a75477-a0ab-4334-a790-b288a837ac34 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:00.801950 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-ea7f83ed-8cda-42dd-9e71-45fdac449b1e req-9d5c0fd1-0208-42fe-9ddb-ccd87bfbbe36 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "13da005a-e085-43aa-8a18-1fe85e4e95af", "name": "network-vif-deleted", "tag": "7b725b13-befd-4a14-98cc-9d0e599f5e17"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:00.814896 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-ea7f83ed-8cda-42dd-9e71-45fdac449b1e req-9d5c0fd1-0208-42fe-9ddb-ccd87bfbbe36 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:00.815443 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-ea7f83ed-8cda-42dd-9e71-45fdac449b1e req-9d5c0fd1-0208-42fe-9ddb-ccd87bfbbe36 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:00.830565 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-ea7f83ed-8cda-42dd-9e71-45fdac449b1e req-9d5c0fd1-0208-42fe-9ddb-ccd87bfbbe36 service nova] Creating event network-vif-deleted:7b725b13-befd-4a14-98cc-9d0e599f5e17 for instance 13da005a-e085-43aa-8a18-1fe85e4e95af on hv-cinder-maste Jul 11 14:45:00.840092 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-ea7f83ed-8cda-42dd-9e71-45fdac449b1e req-9d5c0fd1-0208-42fe-9ddb-ccd87bfbbe36 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041366 Jul 11 14:45:00.840635 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 341/665] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:45:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:45:01.046038 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d6a75477-a0ab-4334-a790-b288a837ac34 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:01.055017 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d6a75477-a0ab-4334-a790-b288a837ac34 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1750 microversion: 2.1 time: 0.279273 Jul 11 14:45:01.055770 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 325/666] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:00 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1750 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:01.374742 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1fbc3c9c-e6c8-450c-8b25-0528506098ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:01.375499 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1fbc3c9c-e6c8-450c-8b25-0528506098ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:01.383807 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1fbc3c9c-e6c8-450c-8b25-0528506098ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:01.384555 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1fbc3c9c-e6c8-450c-8b25-0528506098ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:01.385487 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1fbc3c9c-e6c8-450c-8b25-0528506098ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:01.386123 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1fbc3c9c-e6c8-450c-8b25-0528506098ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:01.406078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-1fbc3c9c-e6c8-450c-8b25-0528506098ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] HTTP exception thrown: Instance 13da005a-e085-43aa-8a18-1fe85e4e95af could not be found. Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1fbc3c9c-e6c8-450c-8b25-0528506098ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Request method failure captured: Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af HTTP/1.1 Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZaOAL7MvbNmMlJTmhhc7VGRk0MBADDiwy8kC6imnZOcaUe8ctAokL2hWFJNSS55xMf5CW1zXWCW5vLH7KcheFILUFqOHdEf_mC24FyMmscF5Poi9nzdCgQxjRRdkaLsfLErjr9x83SftPYUYpu9j4q5t9G2eEOKGcBWW-eUh6A7O8 Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Openstack-Nova-Api-Version: 2.20 Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: f6606b37accd43619058aef23f10745b Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1618811608 Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:45:01.408383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:45:01.409793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/f6606b37accd43619058aef23f10745b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/f6606b37accd43619058aef23f10745b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_f6606b37accd43619058aef23f10745b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/f6606b37accd43619058aef23f10745b"}]}] Jul 11 14:45:01.409793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1618811608 Jul 11 14:45:01.409793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: f6606b37accd43619058aef23f10745b Jul 11 14:45:01.409793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1618811608 Jul 11 14:45:01.409793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-AttachVolumeShelveTestJSON-1618811608-project Jul 11 14:45:01.409793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:45:01.409793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:45:01.409793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 99a62e49e6d249c2a169ef7122e762ec Jul 11 14:45:01.409793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1618811608-project Jul 11 14:45:01.409793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:45:01.409793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance 13da005a-e085-43aa-8a18-1fe85e4e95af could not be found. Jul 11 14:45:01.409793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '13da005a-e085-43aa-8a18-1fe85e4e95af'} Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 13da005a-e085-43aa-8a18-1fe85e4e95af could not be found. Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:45:01.410568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:45:01.411744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:45:01.411744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:45:01.411744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:45:01.411744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:45:01.411744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:45:01.411744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:45:01.411744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:45:01.411744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 13da005a-e085-43aa-8a18-1fe85e4e95af could not be found. Jul 11 14:45:01.411744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:01.411744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:45:01.411744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:01.411744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:01.411744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:45:01.411744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:45:01.411744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:45:01.411744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:45:01.411744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 13da005a-e085-43aa-8a18-1fe85e4e95af could not be found. Jul 11 14:45:01.411744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:01.414260 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1fbc3c9c-e6c8-450c-8b25-0528506098ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Returning 404 to user: Instance 13da005a-e085-43aa-8a18-1fe85e4e95af could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:45:01.415372 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1fbc3c9c-e6c8-450c-8b25-0528506098ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 404 len: 111 microversion: 2.20 time: 0.046950 Jul 11 14:45:01.416161 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 342/667] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:45:01 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 11 14:45:01.426546 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e8e906a8-3027-4a27-83f8-b15b08a1f825 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '.version_select at 0x7f59300613a0>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:01.456985 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e8e906a8-3027-4a27-83f8-b15b08a1f825 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-258714009" status: 204 len: 0 microversion: 2.20 time: 0.035710 Jul 11 14:45:01.458199 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 326/668] 10.222.0.72 () {60 vars in 1258 bytes} [Mon Jul 11 14:45:01 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-258714009 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 11 14:45:02.074320 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-38ced205-6418-4f80-a0dd-0bd3172f2458 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:02.075008 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-38ced205-6418-4f80-a0dd-0bd3172f2458 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:02.081117 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-38ced205-6418-4f80-a0dd-0bd3172f2458 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:02.081807 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-38ced205-6418-4f80-a0dd-0bd3172f2458 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:02.082596 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-38ced205-6418-4f80-a0dd-0bd3172f2458 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:02.083035 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-38ced205-6418-4f80-a0dd-0bd3172f2458 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:02.257579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-b23b6a17-10f6-4432-929f-b84f3d7ee339 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Action: 'create', calling method: .version_select at 0x7f593019e160>, body: {"keypair": {"name": "tempest-keypair-546882754", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBIB3M4M1BJq5k3rfGR3a5Kfs4SgB05eaS8ncKaWImbWr7IQ9O3xby48pFcdDaFZM8yau6YxdugfbeCeFTo4JuhHyCi8deiDfHZnwiHvRrGHE3YVGNgYLaofxiDf+9Cbzw=="}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:02.274306 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-38ced205-6418-4f80-a0dd-0bd3172f2458 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:02.276405 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-b23b6a17-10f6-4432-929f-b84f3d7ee339 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Getting quotas for project f6606b37accd43619058aef23f10745b. Resources: {'key_pairs'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:45:02.280831 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-b23b6a17-10f6-4432-929f-b84f3d7ee339 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Getting quotas for user 99a62e49e6d249c2a169ef7122e762ec and project f6606b37accd43619058aef23f10745b. Resources: {'key_pairs'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:45:02.282460 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-38ced205-6418-4f80-a0dd-0bd3172f2458 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1755 microversion: 2.1 time: 0.212516 Jul 11 14:45:02.283031 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 343/669] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:02 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1755 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:02.309913 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-b23b6a17-10f6-4432-929f-b84f3d7ee339 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.056621 Jul 11 14:45:02.310543 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 327/670] 10.222.0.72 () {62 vars in 1199 bytes} [Mon Jul 11 14:45:02 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 58 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Jul 11 14:45:03.300622 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8e997738-5841-4000-87d1-a5727cec1498 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:03.301238 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8e997738-5841-4000-87d1-a5727cec1498 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:03.304373 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=55,nova_api:UPDATE=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:03.310298 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8e997738-5841-4000-87d1-a5727cec1498 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:03.310792 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8e997738-5841-4000-87d1-a5727cec1498 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:03.311634 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8e997738-5841-4000-87d1-a5727cec1498 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:03.312019 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8e997738-5841-4000-87d1-a5727cec1498 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:03.682030 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8e997738-5841-4000-87d1-a5727cec1498 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:03.690041 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8e997738-5841-4000-87d1-a5727cec1498 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1755 microversion: 2.1 time: 0.393320 Jul 11 14:45:03.690661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 344/671] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:03 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1755 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:03.824803 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-6b618e6b-ec70-4b0c-a0f0-d7c81f41ddaa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1813051592", "imageRef": "a588fa24-23b5-4e53-87c6-7e600ee3a5ef", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "164cf2a3-5444-4c03-8807-160bd45f2610"}], "security_groups": [{"name": "tempest-securitygroup--410850084"}], "key_name": "tempest-keypair-546882754", "user_data": "}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:03.929568 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.network.neutron [None req-6b618e6b-ec70-4b0c-a0f0-d7c81f41ddaa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] validate_networks() for [('164cf2a3-5444-4c03-8807-160bd45f2610', None, None, None, None, None)] {{(pid=130160) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:45:04.538769 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-6b618e6b-ec70-4b0c-a0f0-d7c81f41ddaa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Flavor limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:45:04.539190 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-6b618e6b-ec70-4b0c-a0f0-d7c81f41ddaa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Image limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:45:04.539656 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-6b618e6b-ec70-4b0c-a0f0-d7c81f41ddaa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Flavor pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:45:04.540060 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-6b618e6b-ec70-4b0c-a0f0-d7c81f41ddaa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Image pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:45:04.540471 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-6b618e6b-ec70-4b0c-a0f0-d7c81f41ddaa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:45:04.549445 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6b618e6b-ec70-4b0c-a0f0-d7c81f41ddaa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:04.549888 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6b618e6b-ec70-4b0c-a0f0-d7c81f41ddaa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:04.550709 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6b618e6b-ec70-4b0c-a0f0-d7c81f41ddaa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:04.551139 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6b618e6b-ec70-4b0c-a0f0-d7c81f41ddaa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:04.570465 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-6b618e6b-ec70-4b0c-a0f0-d7c81f41ddaa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Getting quotas for project f6606b37accd43619058aef23f10745b. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:45:04.574694 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-6b618e6b-ec70-4b0c-a0f0-d7c81f41ddaa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Getting quotas for user 99a62e49e6d249c2a169ef7122e762ec and project f6606b37accd43619058aef23f10745b. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:45:04.586376 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-6b618e6b-ec70-4b0c-a0f0-d7c81f41ddaa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Going to run 1 instances... {{(pid=130160) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:45:04.588653 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-6b618e6b-ec70-4b0c-a0f0-d7c81f41ddaa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a588fa24-23b5-4e53-87c6-7e600ee3a5ef',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130160) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:45:04.637605 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-6b618e6b-ec70-4b0c-a0f0-d7c81f41ddaa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 0.817267 Jul 11 14:45:04.638263 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 328/672] 10.222.0.72 () {62 vars in 1187 bytes} [Mon Jul 11 14:45:03 2022] POST /compute/v2.1/servers => generated 401 bytes in 819 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Jul 11 14:45:04.649053 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-bac4d734-1101-4d2d-9df3-00d9634b9ba6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:04.649662 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-bac4d734-1101-4d2d-9df3-00d9634b9ba6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:04.708269 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-bac4d734-1101-4d2d-9df3-00d9634b9ba6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:04.709780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-6f4f7763-e8fd-4a49-9de7-e8755271902d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:04.710459 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-6f4f7763-e8fd-4a49-9de7-e8755271902d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:04.717934 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6f4f7763-e8fd-4a49-9de7-e8755271902d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:04.718393 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6f4f7763-e8fd-4a49-9de7-e8755271902d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:04.719199 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6f4f7763-e8fd-4a49-9de7-e8755271902d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:04.719644 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6f4f7763-e8fd-4a49-9de7-e8755271902d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:04.730662 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-bac4d734-1101-4d2d-9df3-00d9634b9ba6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:04.733072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-bac4d734-1101-4d2d-9df3-00d9634b9ba6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:04.734580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-bac4d734-1101-4d2d-9df3-00d9634b9ba6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1281 microversion: 2.20 time: 0.089576 Jul 11 14:45:04.735808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 345/673] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:45:04 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1281 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:45:04.873727 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6f4f7763-e8fd-4a49-9de7-e8755271902d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:04.883646 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-6f4f7763-e8fd-4a49-9de7-e8755271902d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1755 microversion: 2.1 time: 0.178968 Jul 11 14:45:04.884146 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 329/674] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:04 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1755 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:45:05.753523 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-7ce32cde-a147-41f8-ba0c-f2f677dd89d8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:05.754816 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-7ce32cde-a147-41f8-ba0c-f2f677dd89d8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:05.762841 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7ce32cde-a147-41f8-ba0c-f2f677dd89d8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:05.763327 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7ce32cde-a147-41f8-ba0c-f2f677dd89d8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:05.764106 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7ce32cde-a147-41f8-ba0c-f2f677dd89d8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:05.764545 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7ce32cde-a147-41f8-ba0c-f2f677dd89d8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:05.836508 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7ce32cde-a147-41f8-ba0c-f2f677dd89d8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:05.844424 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7ce32cde-a147-41f8-ba0c-f2f677dd89d8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:05.845935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7ce32cde-a147-41f8-ba0c-f2f677dd89d8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:05.847058 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-7ce32cde-a147-41f8-ba0c-f2f677dd89d8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1351 microversion: 2.20 time: 0.097697 Jul 11 14:45:05.847617 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 346/675] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:45:05 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1351 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:45:05.900940 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-6c982041-c07f-4c4e-871e-b9c29907941c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:05.901519 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-6c982041-c07f-4c4e-871e-b9c29907941c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:05.908240 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6c982041-c07f-4c4e-871e-b9c29907941c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:05.908689 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6c982041-c07f-4c4e-871e-b9c29907941c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:05.909457 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6c982041-c07f-4c4e-871e-b9c29907941c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:05.909897 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6c982041-c07f-4c4e-871e-b9c29907941c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:06.120385 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6c982041-c07f-4c4e-871e-b9c29907941c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:06.128893 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-6c982041-c07f-4c4e-871e-b9c29907941c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1755 microversion: 2.1 time: 0.232367 Jul 11 14:45:06.129493 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 330/676] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:05 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1755 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:06.866455 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-739a9a99-9870-4644-b02c-8a6c693592ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:06.867072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-739a9a99-9870-4644-b02c-8a6c693592ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:06.873779 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-739a9a99-9870-4644-b02c-8a6c693592ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:06.874240 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-739a9a99-9870-4644-b02c-8a6c693592ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:06.875048 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-739a9a99-9870-4644-b02c-8a6c693592ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:06.875543 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-739a9a99-9870-4644-b02c-8a6c693592ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:07.031895 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-739a9a99-9870-4644-b02c-8a6c693592ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:07.040484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-739a9a99-9870-4644-b02c-8a6c693592ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:07.042392 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-739a9a99-9870-4644-b02c-8a6c693592ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:07.043808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-739a9a99-9870-4644-b02c-8a6c693592ca tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1406 microversion: 2.20 time: 0.181798 Jul 11 14:45:07.044543 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 347/677] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:45:06 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1406 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:45:07.147108 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f3e5133c-d10e-4fb2-8166-7e8169a8f09e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:07.147821 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f3e5133c-d10e-4fb2-8166-7e8169a8f09e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:07.154379 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f3e5133c-d10e-4fb2-8166-7e8169a8f09e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:07.154973 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f3e5133c-d10e-4fb2-8166-7e8169a8f09e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:07.155834 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f3e5133c-d10e-4fb2-8166-7e8169a8f09e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:07.156340 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f3e5133c-d10e-4fb2-8166-7e8169a8f09e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:07.361175 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-fe4812a0-975e-4abe-af54-93909e52622f req-d0aabc24-3bdd-4a17-86a2-759e1594877d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "43772fe7-b391-4878-bbea-658602d822ad", "tag": "35522aa1-427b-4c21-88e8-ce45535459a0"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:07.375659 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-fe4812a0-975e-4abe-af54-93909e52622f req-d0aabc24-3bdd-4a17-86a2-759e1594877d service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:07.376095 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-fe4812a0-975e-4abe-af54-93909e52622f req-d0aabc24-3bdd-4a17-86a2-759e1594877d service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:07.388513 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-fe4812a0-975e-4abe-af54-93909e52622f req-d0aabc24-3bdd-4a17-86a2-759e1594877d service nova] Creating event network-changed:35522aa1-427b-4c21-88e8-ce45535459a0 for instance 43772fe7-b391-4878-bbea-658602d822ad on hv-cinder-maste Jul 11 14:45:07.401706 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-fe4812a0-975e-4abe-af54-93909e52622f req-d0aabc24-3bdd-4a17-86a2-759e1594877d service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045236 Jul 11 14:45:07.402608 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 348/678] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:45:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:45:07.426968 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f3e5133c-d10e-4fb2-8166-7e8169a8f09e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:07.438284 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f3e5133c-d10e-4fb2-8166-7e8169a8f09e tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1755 microversion: 2.1 time: 0.295196 Jul 11 14:45:07.438919 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 331/679] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:07 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1755 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:08.064029 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-dd77ab31-0c94-41ef-b9d6-67da926c4de9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:08.064728 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-dd77ab31-0c94-41ef-b9d6-67da926c4de9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:08.071846 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-dd77ab31-0c94-41ef-b9d6-67da926c4de9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:08.072293 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-dd77ab31-0c94-41ef-b9d6-67da926c4de9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:08.073085 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-dd77ab31-0c94-41ef-b9d6-67da926c4de9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:08.073522 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-dd77ab31-0c94-41ef-b9d6-67da926c4de9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:08.201515 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-dd77ab31-0c94-41ef-b9d6-67da926c4de9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:08.210379 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-dd77ab31-0c94-41ef-b9d6-67da926c4de9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:08.211917 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-dd77ab31-0c94-41ef-b9d6-67da926c4de9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:08.213025 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-dd77ab31-0c94-41ef-b9d6-67da926c4de9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1406 microversion: 2.20 time: 0.153907 Jul 11 14:45:08.213573 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 349/680] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:45:08 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1406 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:45:08.456088 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-61b19d44-0595-4ba4-bc7e-ff44a184527d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:08.456857 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-61b19d44-0595-4ba4-bc7e-ff44a184527d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:08.463434 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-61b19d44-0595-4ba4-bc7e-ff44a184527d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:08.463881 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-61b19d44-0595-4ba4-bc7e-ff44a184527d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:08.464640 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-61b19d44-0595-4ba4-bc7e-ff44a184527d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:08.465078 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-61b19d44-0595-4ba4-bc7e-ff44a184527d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:08.610289 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-61b19d44-0595-4ba4-bc7e-ff44a184527d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:08.618906 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-61b19d44-0595-4ba4-bc7e-ff44a184527d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1755 microversion: 2.1 time: 0.166702 Jul 11 14:45:08.619481 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 332/681] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:08 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1755 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:09.230958 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3eb7e970-9a87-4756-87d0-331742381839 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:09.231576 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3eb7e970-9a87-4756-87d0-331742381839 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:09.237975 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3eb7e970-9a87-4756-87d0-331742381839 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:09.238421 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3eb7e970-9a87-4756-87d0-331742381839 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:09.239220 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3eb7e970-9a87-4756-87d0-331742381839 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:09.239660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3eb7e970-9a87-4756-87d0-331742381839 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:09.258895 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:09.381241 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3eb7e970-9a87-4756-87d0-331742381839 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:09.389333 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3eb7e970-9a87-4756-87d0-331742381839 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:09.390883 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3eb7e970-9a87-4756-87d0-331742381839 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:09.391983 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3eb7e970-9a87-4756-87d0-331742381839 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1406 microversion: 2.20 time: 0.164929 Jul 11 14:45:09.392543 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 350/682] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:45:09 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1406 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:45:09.636471 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f39ab86d-760c-40e6-be25-cedec1b95a47 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:09.636999 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f39ab86d-760c-40e6-be25-cedec1b95a47 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:09.643718 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f39ab86d-760c-40e6-be25-cedec1b95a47 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:09.644173 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f39ab86d-760c-40e6-be25-cedec1b95a47 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:09.644944 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f39ab86d-760c-40e6-be25-cedec1b95a47 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:09.645383 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f39ab86d-760c-40e6-be25-cedec1b95a47 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:09.797921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f39ab86d-760c-40e6-be25-cedec1b95a47 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:09.806145 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f39ab86d-760c-40e6-be25-cedec1b95a47 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1755 microversion: 2.1 time: 0.173633 Jul 11 14:45:09.806760 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 333/683] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:09 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1755 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:10.410794 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-28ba147b-33ce-4173-8820-7f4480705824 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:10.412042 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-28ba147b-33ce-4173-8820-7f4480705824 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:10.419543 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-28ba147b-33ce-4173-8820-7f4480705824 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:10.420000 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-28ba147b-33ce-4173-8820-7f4480705824 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:10.420751 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-28ba147b-33ce-4173-8820-7f4480705824 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:10.421181 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-28ba147b-33ce-4173-8820-7f4480705824 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:10.556361 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-28ba147b-33ce-4173-8820-7f4480705824 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:10.564547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-28ba147b-33ce-4173-8820-7f4480705824 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:10.566058 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-28ba147b-33ce-4173-8820-7f4480705824 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:10.567184 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-28ba147b-33ce-4173-8820-7f4480705824 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1406 microversion: 2.20 time: 0.160512 Jul 11 14:45:10.567734 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 351/684] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:45:10 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1406 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:45:10.823602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f73c1acb-a511-4fdf-bfab-6aaf2b915509 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:10.824335 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f73c1acb-a511-4fdf-bfab-6aaf2b915509 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:10.830603 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f73c1acb-a511-4fdf-bfab-6aaf2b915509 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:10.831049 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f73c1acb-a511-4fdf-bfab-6aaf2b915509 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:10.832331 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f73c1acb-a511-4fdf-bfab-6aaf2b915509 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:10.832766 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f73c1acb-a511-4fdf-bfab-6aaf2b915509 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:10.978027 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f73c1acb-a511-4fdf-bfab-6aaf2b915509 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:10.988147 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f73c1acb-a511-4fdf-bfab-6aaf2b915509 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1755 microversion: 2.1 time: 0.168125 Jul 11 14:45:10.988746 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 334/685] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:10 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1755 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:45:11.585692 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-7064e905-8940-4195-8c89-60a69236fc03 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:11.586290 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-7064e905-8940-4195-8c89-60a69236fc03 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:11.593321 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7064e905-8940-4195-8c89-60a69236fc03 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:11.593775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7064e905-8940-4195-8c89-60a69236fc03 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:11.594605 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7064e905-8940-4195-8c89-60a69236fc03 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:11.595049 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7064e905-8940-4195-8c89-60a69236fc03 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:11.739960 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7064e905-8940-4195-8c89-60a69236fc03 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:11.749032 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7064e905-8940-4195-8c89-60a69236fc03 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:11.751521 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7064e905-8940-4195-8c89-60a69236fc03 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:11.752414 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-7064e905-8940-4195-8c89-60a69236fc03 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1406 microversion: 2.20 time: 0.170269 Jul 11 14:45:11.752996 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 352/686] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:45:11 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1406 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:45:11.828681 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-eb72edbf-5553-4bee-9c70-50d8c7a102bb req-4b2b1526-f0de-4dde-8286-21b259b6193f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "43772fe7-b391-4878-bbea-658602d822ad", "name": "network-vif-plugged", "status": "completed", "tag": "35522aa1-427b-4c21-88e8-ce45535459a0"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:11.839992 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-eb72edbf-5553-4bee-9c70-50d8c7a102bb req-4b2b1526-f0de-4dde-8286-21b259b6193f service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:11.840846 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-eb72edbf-5553-4bee-9c70-50d8c7a102bb req-4b2b1526-f0de-4dde-8286-21b259b6193f service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:11.853151 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-eb72edbf-5553-4bee-9c70-50d8c7a102bb req-4b2b1526-f0de-4dde-8286-21b259b6193f service nova] Creating event network-vif-plugged:35522aa1-427b-4c21-88e8-ce45535459a0 for instance 43772fe7-b391-4878-bbea-658602d822ad on hv-cinder-maste Jul 11 14:45:11.858375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-eb72edbf-5553-4bee-9c70-50d8c7a102bb req-4b2b1526-f0de-4dde-8286-21b259b6193f service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033420 Jul 11 14:45:11.859361 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 335/687] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:45:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:45:12.007310 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8c7ed54a-ccb0-4c14-80ad-babd9ea3ffec tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:12.007970 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8c7ed54a-ccb0-4c14-80ad-babd9ea3ffec tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:12.014322 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8c7ed54a-ccb0-4c14-80ad-babd9ea3ffec tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:12.014619 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8c7ed54a-ccb0-4c14-80ad-babd9ea3ffec tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:12.015199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8c7ed54a-ccb0-4c14-80ad-babd9ea3ffec tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:12.015636 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8c7ed54a-ccb0-4c14-80ad-babd9ea3ffec tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:12.161175 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8c7ed54a-ccb0-4c14-80ad-babd9ea3ffec tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:12.169921 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8c7ed54a-ccb0-4c14-80ad-babd9ea3ffec tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1755 microversion: 2.1 time: 0.166673 Jul 11 14:45:12.170522 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 353/688] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:12 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1755 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:12.771338 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-db817859-a221-48e1-a279-4b0b0ac84a77 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:12.772047 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-db817859-a221-48e1-a279-4b0b0ac84a77 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:12.778442 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-db817859-a221-48e1-a279-4b0b0ac84a77 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:12.778903 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-db817859-a221-48e1-a279-4b0b0ac84a77 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:12.779663 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-db817859-a221-48e1-a279-4b0b0ac84a77 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:12.780103 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-db817859-a221-48e1-a279-4b0b0ac84a77 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:13.070409 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-db817859-a221-48e1-a279-4b0b0ac84a77 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:13.078382 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-db817859-a221-48e1-a279-4b0b0ac84a77 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:13.079922 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-db817859-a221-48e1-a279-4b0b0ac84a77 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:13.081022 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-db817859-a221-48e1-a279-4b0b0ac84a77 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1406 microversion: 2.20 time: 0.314332 Jul 11 14:45:13.081837 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 336/689] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:45:12 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1406 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:45:13.186314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d7c6ae05-2731-480f-b8c6-b343837b8354 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:13.186950 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d7c6ae05-2731-480f-b8c6-b343837b8354 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:13.194501 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d7c6ae05-2731-480f-b8c6-b343837b8354 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:13.194956 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d7c6ae05-2731-480f-b8c6-b343837b8354 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:13.195741 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d7c6ae05-2731-480f-b8c6-b343837b8354 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:13.196185 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d7c6ae05-2731-480f-b8c6-b343837b8354 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:13.395846 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d7c6ae05-2731-480f-b8c6-b343837b8354 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:13.405869 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-d7c6ae05-2731-480f-b8c6-b343837b8354 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1755 microversion: 2.1 time: 0.223355 Jul 11 14:45:13.407224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 354/690] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:13 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1755 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:14.099130 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3d6df5f8-1a26-4f95-9510-30015dd63bc5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:14.100385 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-3d6df5f8-1a26-4f95-9510-30015dd63bc5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:14.102644 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=73,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:14.107920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d6df5f8-1a26-4f95-9510-30015dd63bc5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:14.108370 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d6df5f8-1a26-4f95-9510-30015dd63bc5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:14.109141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d6df5f8-1a26-4f95-9510-30015dd63bc5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:14.109581 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d6df5f8-1a26-4f95-9510-30015dd63bc5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:14.243830 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-3d6df5f8-1a26-4f95-9510-30015dd63bc5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:14.251792 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-3d6df5f8-1a26-4f95-9510-30015dd63bc5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:14.253356 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-3d6df5f8-1a26-4f95-9510-30015dd63bc5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:14.254525 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-3d6df5f8-1a26-4f95-9510-30015dd63bc5 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1597 microversion: 2.20 time: 0.159383 Jul 11 14:45:14.255074 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 337/691] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:45:14 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1597 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:45:14.271658 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d94d424e-1c63-4639-90e8-9bd87b31a80c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:14.272150 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d94d424e-1c63-4639-90e8-9bd87b31a80c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:14.278592 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d94d424e-1c63-4639-90e8-9bd87b31a80c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:14.279041 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d94d424e-1c63-4639-90e8-9bd87b31a80c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:14.279791 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d94d424e-1c63-4639-90e8-9bd87b31a80c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:14.280219 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d94d424e-1c63-4639-90e8-9bd87b31a80c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:14.361169 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-d94d424e-1c63-4639-90e8-9bd87b31a80c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.093380 Jul 11 14:45:14.361768 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 355/692] 10.222.0.72 () {60 vars in 1315 bytes} [Mon Jul 11 14:45:14 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad/os-interface => generated 284 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 11 14:45:14.423736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-30ba31a9-3f17-4ccd-bfcd-9794d6a2b752 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:14.424364 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-30ba31a9-3f17-4ccd-bfcd-9794d6a2b752 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:14.431359 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-30ba31a9-3f17-4ccd-bfcd-9794d6a2b752 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:14.431839 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-30ba31a9-3f17-4ccd-bfcd-9794d6a2b752 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:14.432621 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-30ba31a9-3f17-4ccd-bfcd-9794d6a2b752 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:14.433069 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-30ba31a9-3f17-4ccd-bfcd-9794d6a2b752 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:14.435831 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:14.560407 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=2 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:14.579832 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-30ba31a9-3f17-4ccd-bfcd-9794d6a2b752 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:14.582277 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=10 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:14.590064 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-30ba31a9-3f17-4ccd-bfcd-9794d6a2b752 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1755 microversion: 2.1 time: 0.170179 Jul 11 14:45:14.590701 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 338/693] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:14 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1755 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:15.294020 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=2 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:15.298666 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-8dd5c101-d5fb-4237-9229-aac53d9ff6bb req-a18d1d52-6912-4e62-b680-578cae88128b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "43772fe7-b391-4878-bbea-658602d822ad", "tag": "35522aa1-427b-4c21-88e8-ce45535459a0"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:15.307646 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-8dd5c101-d5fb-4237-9229-aac53d9ff6bb req-a18d1d52-6912-4e62-b680-578cae88128b service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:15.307942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-8dd5c101-d5fb-4237-9229-aac53d9ff6bb req-a18d1d52-6912-4e62-b680-578cae88128b service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:15.319866 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-8dd5c101-d5fb-4237-9229-aac53d9ff6bb req-a18d1d52-6912-4e62-b680-578cae88128b service nova] Creating event network-changed:35522aa1-427b-4c21-88e8-ce45535459a0 for instance 43772fe7-b391-4878-bbea-658602d822ad on hv-cinder-maste Jul 11 14:45:15.325105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-8dd5c101-d5fb-4237-9229-aac53d9ff6bb req-a18d1d52-6912-4e62-b680-578cae88128b service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031614 Jul 11 14:45:15.325709 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 356/694] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:45:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:45:15.609983 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-68cfd3fc-7848-4674-9e29-c0fdfcc789a5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:15.611074 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-68cfd3fc-7848-4674-9e29-c0fdfcc789a5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:15.619254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-68cfd3fc-7848-4674-9e29-c0fdfcc789a5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:15.619773 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-68cfd3fc-7848-4674-9e29-c0fdfcc789a5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:15.620566 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-68cfd3fc-7848-4674-9e29-c0fdfcc789a5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:15.621780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-68cfd3fc-7848-4674-9e29-c0fdfcc789a5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:15.856785 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-68cfd3fc-7848-4674-9e29-c0fdfcc789a5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:15.865248 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-68cfd3fc-7848-4674-9e29-c0fdfcc789a5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1755 microversion: 2.1 time: 0.260150 Jul 11 14:45:15.865975 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 339/695] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:15 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1755 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:16.884719 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9e1d00f6-c5b9-4d0b-beea-580f43de8b1c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:16.885490 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9e1d00f6-c5b9-4d0b-beea-580f43de8b1c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:16.891785 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9e1d00f6-c5b9-4d0b-beea-580f43de8b1c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:16.892254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9e1d00f6-c5b9-4d0b-beea-580f43de8b1c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:16.893032 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9e1d00f6-c5b9-4d0b-beea-580f43de8b1c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:16.893475 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9e1d00f6-c5b9-4d0b-beea-580f43de8b1c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:16.987304 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-0422ddc4-2d62-4064-86e0-5f7ff6c1c96c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:16.987964 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-0422ddc4-2d62-4064-86e0-5f7ff6c1c96c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:16.995754 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0422ddc4-2d62-4064-86e0-5f7ff6c1c96c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:16.996493 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0422ddc4-2d62-4064-86e0-5f7ff6c1c96c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:16.997280 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0422ddc4-2d62-4064-86e0-5f7ff6c1c96c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:16.997725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0422ddc4-2d62-4064-86e0-5f7ff6c1c96c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:17.024076 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-0422ddc4-2d62-4064-86e0-5f7ff6c1c96c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Going to try to terminate instance {{(pid=130160) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:45:17.049215 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9e1d00f6-c5b9-4d0b-beea-580f43de8b1c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:17.058868 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9e1d00f6-c5b9-4d0b-beea-580f43de8b1c tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1755 microversion: 2.1 time: 0.179416 Jul 11 14:45:17.059463 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 357/696] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:16 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1755 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:17.106988 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-0422ddc4-2d62-4064-86e0-5f7ff6c1c96c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "DELETE /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 204 len: 0 microversion: 2.1 time: 0.123119 Jul 11 14:45:17.108685 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 340/697] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:45:16 2022] DELETE /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 11 14:45:17.118174 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f81a3502-e2de-40ac-b537-7beeebbcb1e8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:17.119122 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f81a3502-e2de-40ac-b537-7beeebbcb1e8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:17.125183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f81a3502-e2de-40ac-b537-7beeebbcb1e8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:17.125709 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f81a3502-e2de-40ac-b537-7beeebbcb1e8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:17.126569 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f81a3502-e2de-40ac-b537-7beeebbcb1e8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:17.127076 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f81a3502-e2de-40ac-b537-7beeebbcb1e8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:17.257696 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f81a3502-e2de-40ac-b537-7beeebbcb1e8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:17.265572 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f81a3502-e2de-40ac-b537-7beeebbcb1e8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1578 microversion: 2.1 time: 0.151835 Jul 11 14:45:17.266131 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 358/698] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:17 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1578 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:18.078555 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-4cdbbcbc-f2d3-4599-9d17-75a5197b9ce5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:18.079281 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-4cdbbcbc-f2d3-4599-9d17-75a5197b9ce5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:18.086571 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4cdbbcbc-f2d3-4599-9d17-75a5197b9ce5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:18.087017 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4cdbbcbc-f2d3-4599-9d17-75a5197b9ce5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:18.087776 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4cdbbcbc-f2d3-4599-9d17-75a5197b9ce5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:18.088205 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4cdbbcbc-f2d3-4599-9d17-75a5197b9ce5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:18.232257 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4cdbbcbc-f2d3-4599-9d17-75a5197b9ce5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:18.240347 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-4cdbbcbc-f2d3-4599-9d17-75a5197b9ce5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1755 microversion: 2.1 time: 0.166047 Jul 11 14:45:18.240925 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 341/699] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:18 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1755 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:18.282219 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-93980bb6-a50e-4467-affa-4814c39845b8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:18.282959 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-93980bb6-a50e-4467-affa-4814c39845b8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:18.290349 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-93980bb6-a50e-4467-affa-4814c39845b8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:18.290862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-93980bb6-a50e-4467-affa-4814c39845b8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:18.291683 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-93980bb6-a50e-4467-affa-4814c39845b8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:18.292169 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-93980bb6-a50e-4467-affa-4814c39845b8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:18.426105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-93980bb6-a50e-4467-affa-4814c39845b8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:18.434253 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-93980bb6-a50e-4467-affa-4814c39845b8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1578 microversion: 2.1 time: 0.155824 Jul 11 14:45:18.434861 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 359/700] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:18 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1578 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:19.148062 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3abb0dc2-4202-4592-a1d2-eef64e55df34 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:19.148998 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-3abb0dc2-4202-4592-a1d2-eef64e55df34 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:19.155535 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3abb0dc2-4202-4592-a1d2-eef64e55df34 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:19.155983 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3abb0dc2-4202-4592-a1d2-eef64e55df34 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:19.156743 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3abb0dc2-4202-4592-a1d2-eef64e55df34 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:19.157188 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3abb0dc2-4202-4592-a1d2-eef64e55df34 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:19.186103 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-3abb0dc2-4202-4592-a1d2-eef64e55df34 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.041931 Jul 11 14:45:19.186683 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 342/701] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:45:19 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d/os-volume_attachments => generated 25 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 11 14:45:19.244564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f4f68d64-c9f4-4cd2-8d87-368bdc93ac62 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:19.245048 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f4f68d64-c9f4-4cd2-8d87-368bdc93ac62 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:19.251041 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f4f68d64-c9f4-4cd2-8d87-368bdc93ac62 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:19.251484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f4f68d64-c9f4-4cd2-8d87-368bdc93ac62 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:19.252340 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f4f68d64-c9f4-4cd2-8d87-368bdc93ac62 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:19.252773 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f4f68d64-c9f4-4cd2-8d87-368bdc93ac62 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:19.256866 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-dca71d06-cd9b-4e37-8d43-49dde3941f98 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:19.257657 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-dca71d06-cd9b-4e37-8d43-49dde3941f98 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:19.263626 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dca71d06-cd9b-4e37-8d43-49dde3941f98 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:19.264079 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dca71d06-cd9b-4e37-8d43-49dde3941f98 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:19.264875 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dca71d06-cd9b-4e37-8d43-49dde3941f98 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:19.265313 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dca71d06-cd9b-4e37-8d43-49dde3941f98 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:19.277742 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f4f68d64-c9f4-4cd2-8d87-368bdc93ac62 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Going to try to terminate instance {{(pid=130159) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:45:19.357787 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f4f68d64-c9f4-4cd2-8d87-368bdc93ac62 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "DELETE /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 204 len: 0 microversion: 2.1 time: 0.117421 Jul 11 14:45:19.358524 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 360/702] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:45:19 2022] DELETE /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 11 14:45:19.367475 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-27acd9c2-72ce-43ef-92f7-4fa5a077814f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:19.368182 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-27acd9c2-72ce-43ef-92f7-4fa5a077814f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:19.374051 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-27acd9c2-72ce-43ef-92f7-4fa5a077814f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:19.374542 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-27acd9c2-72ce-43ef-92f7-4fa5a077814f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:19.375307 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-27acd9c2-72ce-43ef-92f7-4fa5a077814f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:19.375744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-27acd9c2-72ce-43ef-92f7-4fa5a077814f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:19.410538 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-dca71d06-cd9b-4e37-8d43-49dde3941f98 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:19.419308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-dca71d06-cd9b-4e37-8d43-49dde3941f98 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1758 microversion: 2.1 time: 0.166164 Jul 11 14:45:19.419894 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 343/703] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:19 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1758 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:45:19.452070 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-b89050d9-40d4-4c8f-b76c-dfac7a5a79f5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:19.452690 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-b89050d9-40d4-4c8f-b76c-dfac7a5a79f5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:19.460386 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b89050d9-40d4-4c8f-b76c-dfac7a5a79f5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:19.460918 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b89050d9-40d4-4c8f-b76c-dfac7a5a79f5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:19.461824 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b89050d9-40d4-4c8f-b76c-dfac7a5a79f5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:19.462385 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b89050d9-40d4-4c8f-b76c-dfac7a5a79f5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:19.506255 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-27acd9c2-72ce-43ef-92f7-4fa5a077814f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:19.513925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-27acd9c2-72ce-43ef-92f7-4fa5a077814f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1672 microversion: 2.1 time: 0.151022 Jul 11 14:45:19.514515 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 361/704] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:19 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1672 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:45:19.598545 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-b89050d9-40d4-4c8f-b76c-dfac7a5a79f5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:19.606654 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-b89050d9-40d4-4c8f-b76c-dfac7a5a79f5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1578 microversion: 2.1 time: 0.158396 Jul 11 14:45:19.607220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 344/705] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:19 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1578 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:45:20.531905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-5f5f9467-88c1-464f-b651-a05ebb064ef8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:20.532577 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-5f5f9467-88c1-464f-b651-a05ebb064ef8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:20.540698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5f5f9467-88c1-464f-b651-a05ebb064ef8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:20.541285 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5f5f9467-88c1-464f-b651-a05ebb064ef8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:20.542058 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5f5f9467-88c1-464f-b651-a05ebb064ef8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:20.542524 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5f5f9467-88c1-464f-b651-a05ebb064ef8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:20.624115 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e8720970-8436-4575-b93d-c9e98116b377 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:20.624730 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-e8720970-8436-4575-b93d-c9e98116b377 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:20.631531 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e8720970-8436-4575-b93d-c9e98116b377 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:20.631980 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e8720970-8436-4575-b93d-c9e98116b377 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:20.632787 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e8720970-8436-4575-b93d-c9e98116b377 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:20.633229 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e8720970-8436-4575-b93d-c9e98116b377 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:20.684540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-5f5f9467-88c1-464f-b651-a05ebb064ef8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:20.692203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-5f5f9467-88c1-464f-b651-a05ebb064ef8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1672 microversion: 2.1 time: 0.164623 Jul 11 14:45:20.692785 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 362/706] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:20 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1672 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:20.770284 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e8720970-8436-4575-b93d-c9e98116b377 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:20.778732 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e8720970-8436-4575-b93d-c9e98116b377 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1578 microversion: 2.1 time: 0.158486 Jul 11 14:45:20.779319 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 345/707] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:20 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1578 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:45:21.709973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-7da49195-8765-454d-a3f2-8702b49a9340 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:21.710607 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-7da49195-8765-454d-a3f2-8702b49a9340 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:21.717010 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7da49195-8765-454d-a3f2-8702b49a9340 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:21.717452 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7da49195-8765-454d-a3f2-8702b49a9340 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:21.718230 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7da49195-8765-454d-a3f2-8702b49a9340 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:21.718684 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7da49195-8765-454d-a3f2-8702b49a9340 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:21.795789 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-259ba3ef-ceb8-4fea-8e60-85d7bb8cd7f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:21.797222 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-259ba3ef-ceb8-4fea-8e60-85d7bb8cd7f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:21.805654 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-259ba3ef-ceb8-4fea-8e60-85d7bb8cd7f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:21.806569 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-259ba3ef-ceb8-4fea-8e60-85d7bb8cd7f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:21.807586 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-259ba3ef-ceb8-4fea-8e60-85d7bb8cd7f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:21.808434 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-259ba3ef-ceb8-4fea-8e60-85d7bb8cd7f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:21.948707 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7da49195-8765-454d-a3f2-8702b49a9340 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:21.957572 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-7da49195-8765-454d-a3f2-8702b49a9340 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1672 microversion: 2.1 time: 0.251576 Jul 11 14:45:21.960574 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 363/708] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:21 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1672 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:21.965566 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-1a6ea7aa-4690-45b3-ad46-4731cdde9df5 req-bf7748a4-5a3c-4468-993d-548f1d59635f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3a91aed4-52ff-4220-bc8f-571c50452865", "name": "network-vif-unplugged", "status": "completed", "tag": "957d5045-a8c1-4d28-8ee4-755331ae2c84"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:21.974223 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-1a6ea7aa-4690-45b3-ad46-4731cdde9df5 req-bf7748a4-5a3c-4468-993d-548f1d59635f service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:21.975320 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-1a6ea7aa-4690-45b3-ad46-4731cdde9df5 req-bf7748a4-5a3c-4468-993d-548f1d59635f service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:21.987194 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-1a6ea7aa-4690-45b3-ad46-4731cdde9df5 req-bf7748a4-5a3c-4468-993d-548f1d59635f service nova] Creating event network-vif-unplugged:957d5045-a8c1-4d28-8ee4-755331ae2c84 for instance 3a91aed4-52ff-4220-bc8f-571c50452865 on hv-cinder-maste Jul 11 14:45:21.993600 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-1a6ea7aa-4690-45b3-ad46-4731cdde9df5 req-bf7748a4-5a3c-4468-993d-548f1d59635f service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.031121 Jul 11 14:45:21.994210 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 364/709] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:45:21 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:45:21.999590 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-259ba3ef-ceb8-4fea-8e60-85d7bb8cd7f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:22.007106 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-259ba3ef-ceb8-4fea-8e60-85d7bb8cd7f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1578 microversion: 2.1 time: 0.215204 Jul 11 14:45:22.007682 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 346/710] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:21 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1578 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:45:22.695011 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-cdcb8ca3-3047-4f5d-ad8a-c6ec1befa2eb req-e8c7a4b8-a9f9-4acd-932c-94da2a618fda service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3a91aed4-52ff-4220-bc8f-571c50452865", "name": "network-vif-deleted", "tag": "957d5045-a8c1-4d28-8ee4-755331ae2c84"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:22.718689 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-cdcb8ca3-3047-4f5d-ad8a-c6ec1befa2eb req-e8c7a4b8-a9f9-4acd-932c-94da2a618fda service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:22.724339 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-cdcb8ca3-3047-4f5d-ad8a-c6ec1befa2eb req-e8c7a4b8-a9f9-4acd-932c-94da2a618fda service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:22.738678 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-cdcb8ca3-3047-4f5d-ad8a-c6ec1befa2eb req-e8c7a4b8-a9f9-4acd-932c-94da2a618fda service nova] Creating event network-vif-deleted:957d5045-a8c1-4d28-8ee4-755331ae2c84 for instance 3a91aed4-52ff-4220-bc8f-571c50452865 on hv-cinder-maste Jul 11 14:45:22.747249 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-cdcb8ca3-3047-4f5d-ad8a-c6ec1befa2eb req-e8c7a4b8-a9f9-4acd-932c-94da2a618fda service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057048 Jul 11 14:45:22.762913 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 365/711] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:45:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:45:22.976981 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c762d594-9e66-4862-83b6-78f143e4d71a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:22.978385 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c762d594-9e66-4862-83b6-78f143e4d71a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:22.988962 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c762d594-9e66-4862-83b6-78f143e4d71a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:22.989426 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c762d594-9e66-4862-83b6-78f143e4d71a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:22.990568 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c762d594-9e66-4862-83b6-78f143e4d71a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:22.991008 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c762d594-9e66-4862-83b6-78f143e4d71a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:23.025366 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a07abb88-abbc-48db-a890-81e0145cd48e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:23.025985 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a07abb88-abbc-48db-a890-81e0145cd48e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:23.037799 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a07abb88-abbc-48db-a890-81e0145cd48e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:23.038895 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a07abb88-abbc-48db-a890-81e0145cd48e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:23.039842 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a07abb88-abbc-48db-a890-81e0145cd48e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:23.040441 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a07abb88-abbc-48db-a890-81e0145cd48e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:23.107694 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c762d594-9e66-4862-83b6-78f143e4d71a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:23.118712 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c762d594-9e66-4862-83b6-78f143e4d71a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 200 len: 1606 microversion: 2.1 time: 0.145791 Jul 11 14:45:23.120117 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 347/712] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:22 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 1606 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:23.148096 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a07abb88-abbc-48db-a890-81e0145cd48e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:23.158387 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a07abb88-abbc-48db-a890-81e0145cd48e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 200 len: 1226 microversion: 2.1 time: 0.137711 Jul 11 14:45:23.159404 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 366/713] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:23 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 1226 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:45:23.398466 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-97510ce5-affc-4120-a1da-3559ef46e04b req-1691c5ea-fadc-4ad6-81e8-47bbb3bf1e04 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "62a90c58-0027-47e8-8909-44512060e08d", "name": "network-vif-deleted", "tag": "edfcc9e3-9cd5-4b35-adc9-68a5f06aeb87"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:23.417786 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-97510ce5-affc-4120-a1da-3559ef46e04b req-1691c5ea-fadc-4ad6-81e8-47bbb3bf1e04 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:23.418162 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-97510ce5-affc-4120-a1da-3559ef46e04b req-1691c5ea-fadc-4ad6-81e8-47bbb3bf1e04 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:23.455858 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-97510ce5-affc-4120-a1da-3559ef46e04b req-1691c5ea-fadc-4ad6-81e8-47bbb3bf1e04 service nova] Creating event network-vif-deleted:edfcc9e3-9cd5-4b35-adc9-68a5f06aeb87 for instance 62a90c58-0027-47e8-8909-44512060e08d on hv-cinder-maste Jul 11 14:45:23.467833 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-97510ce5-affc-4120-a1da-3559ef46e04b req-1691c5ea-fadc-4ad6-81e8-47bbb3bf1e04 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.074045 Jul 11 14:45:23.468964 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 348/714] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:45:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:45:24.137055 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-465043e6-9595-4e3f-8719-8135f6be3a4b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:24.137834 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-465043e6-9595-4e3f-8719-8135f6be3a4b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:24.144558 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-465043e6-9595-4e3f-8719-8135f6be3a4b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:24.145007 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-465043e6-9595-4e3f-8719-8135f6be3a4b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:24.145773 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-465043e6-9595-4e3f-8719-8135f6be3a4b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:24.146238 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-465043e6-9595-4e3f-8719-8135f6be3a4b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:24.160927 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-465043e6-9595-4e3f-8719-8135f6be3a4b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] HTTP exception thrown: Instance 62a90c58-0027-47e8-8909-44512060e08d could not be found. Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-465043e6-9595-4e3f-8719-8135f6be3a4b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Request method failure captured: Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d HTTP/1.1 Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZgOjz1xpcRYZZu-Kh_5N3i0wpyKor5piLH_ZAuxS7Y7F4dt1mCndOM5aBnPiEJue8x_XFxSfgCnilWaGVpEbgHnMB6NZWPYBeKDQCtfPPAwXFcEuCxxWlEzENjbPi2j267u3jA32MzeWpRtu42O7GlF56dv-e0bwbLODZDNeFgobk Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: c0e363b1584848c1aecbe186cf91eec5 Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-AttachVolumeTestJSON-588294681 Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:45:24.162775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:45:24.164229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/c0e363b1584848c1aecbe186cf91eec5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/c0e363b1584848c1aecbe186cf91eec5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_c0e363b1584848c1aecbe186cf91eec5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/c0e363b1584848c1aecbe186cf91eec5"}]}] Jul 11 14:45:24.164229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-AttachVolumeTestJSON-588294681 Jul 11 14:45:24.164229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: c0e363b1584848c1aecbe186cf91eec5 Jul 11 14:45:24.164229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-AttachVolumeTestJSON-588294681 Jul 11 14:45:24.164229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-AttachVolumeTestJSON-588294681-project Jul 11 14:45:24.164229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:45:24.164229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:45:24.164229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 467992b53c4749858fd8b0ccd2db2393 Jul 11 14:45:24.164229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-AttachVolumeTestJSON-588294681-project Jul 11 14:45:24.164229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:45:24.164229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance 62a90c58-0027-47e8-8909-44512060e08d could not be found. Jul 11 14:45:24.164229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '62a90c58-0027-47e8-8909-44512060e08d'} Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 62a90c58-0027-47e8-8909-44512060e08d could not be found. Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:45:24.165072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:45:24.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:45:24.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:45:24.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:45:24.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:45:24.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:45:24.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:45:24.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:45:24.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 62a90c58-0027-47e8-8909-44512060e08d could not be found. Jul 11 14:45:24.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:24.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:45:24.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:24.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:24.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:45:24.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:45:24.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:45:24.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:45:24.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 62a90c58-0027-47e8-8909-44512060e08d could not be found. Jul 11 14:45:24.166314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:24.167526 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-465043e6-9595-4e3f-8719-8135f6be3a4b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Returning 404 to user: Instance 62a90c58-0027-47e8-8909-44512060e08d could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:45:24.167526 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-465043e6-9595-4e3f-8719-8135f6be3a4b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 404 len: 111 microversion: 2.1 time: 0.031275 Jul 11 14:45:24.167526 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 367/715] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:24 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 11 14:45:24.174515 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-dcb5a595-3991-48ef-8137-0a7fc2123f97 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '.version_select at 0x7f5930061af0>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:24.182784 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-159a366c-81dd-40a3-bae0-257cd1ac2f8d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:24.183397 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-159a366c-81dd-40a3-bae0-257cd1ac2f8d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:24.190130 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-159a366c-81dd-40a3-bae0-257cd1ac2f8d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:24.190672 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-159a366c-81dd-40a3-bae0-257cd1ac2f8d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:24.191547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-159a366c-81dd-40a3-bae0-257cd1ac2f8d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:24.191961 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-159a366c-81dd-40a3-bae0-257cd1ac2f8d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:24.198232 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-dcb5a595-3991-48ef-8137-0a7fc2123f97 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1843444996" status: 202 len: 0 microversion: 2.1 time: 0.028387 Jul 11 14:45:24.200524 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 349/716] 10.222.0.72 () {58 vars in 1220 bytes} [Mon Jul 11 14:45:24 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1843444996 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:45:24.207701 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-159a366c-81dd-40a3-bae0-257cd1ac2f8d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] HTTP exception thrown: Instance 3a91aed4-52ff-4220-bc8f-571c50452865 could not be found. Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-159a366c-81dd-40a3-bae0-257cd1ac2f8d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Request method failure captured: Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 HTTP/1.1 Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDatkPKLV3gGEijxILG_rdpdg9DW_7qpS_AOU-l0RpuGuc3ntiuQCrsxEAFHVLaJAgdSMGRlHE7aWkLB0HxWtv0M50LGlLeUNDhMrIRsXvtM1cdPVKwP9HSD50o6sO2CtsHMEnTPPPgBI_ekQM6IpcM5Nj1c1ZwFE9vfwHMe24itu1A Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: 3d541b7a72dc4b4494a42bda6fa75026 Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-TestVolumeBootPattern-593501879 Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:45:24.209259 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:45:24.210650 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/3d541b7a72dc4b4494a42bda6fa75026"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/3d541b7a72dc4b4494a42bda6fa75026"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_3d541b7a72dc4b4494a42bda6fa75026"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/3d541b7a72dc4b4494a42bda6fa75026"}]}] Jul 11 14:45:24.210650 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-TestVolumeBootPattern-593501879 Jul 11 14:45:24.210650 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: 3d541b7a72dc4b4494a42bda6fa75026 Jul 11 14:45:24.210650 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-TestVolumeBootPattern-593501879 Jul 11 14:45:24.210650 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-TestVolumeBootPattern-593501879-project Jul 11 14:45:24.210650 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:45:24.210650 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:45:24.210650 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 0f8e300cd96e446db7430044e69d997a Jul 11 14:45:24.210650 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-TestVolumeBootPattern-593501879-project Jul 11 14:45:24.210650 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:45:24.210650 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance 3a91aed4-52ff-4220-bc8f-571c50452865 could not be found. Jul 11 14:45:24.210650 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '3a91aed4-52ff-4220-bc8f-571c50452865'} Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3a91aed4-52ff-4220-bc8f-571c50452865 could not be found. Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:45:24.211406 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:45:24.212564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:45:24.212564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:45:24.212564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:45:24.212564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:45:24.212564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:45:24.212564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:45:24.212564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:45:24.212564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3a91aed4-52ff-4220-bc8f-571c50452865 could not be found. Jul 11 14:45:24.212564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:24.212564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:45:24.212564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:24.212564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:24.212564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:45:24.212564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:45:24.212564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:45:24.212564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:45:24.212564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3a91aed4-52ff-4220-bc8f-571c50452865 could not be found. Jul 11 14:45:24.212564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:24.213619 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-159a366c-81dd-40a3-bae0-257cd1ac2f8d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Returning 404 to user: Instance 3a91aed4-52ff-4220-bc8f-571c50452865 could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:45:24.213619 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-159a366c-81dd-40a3-bae0-257cd1ac2f8d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 404 len: 111 microversion: 2.1 time: 0.031779 Jul 11 14:45:24.213619 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 368/717] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:24 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:45:24.328002 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-23e1f807-b6c1-44f2-a090-e6510d355c6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-2090316506", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-2098849656", "security_groups": [{"name": "tempest-secgroup-smoke-2075789476"}], "block_device_mapping_v2": [{"uuid": "ccbe23b1-2e12-4e18-9982-7c804ff58b5f", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "e7191ae9-3a75-447f-ae05-2827e78eae91"}]}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:24.562343 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.network.neutron [None req-23e1f807-b6c1-44f2-a090-e6510d355c6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] validate_networks() for [('e7191ae9-3a75-447f-ae05-2827e78eae91', None, None, None, None, None)] {{(pid=130160) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:45:25.189324 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-70610fa9-5aae-4cff-a924-42a216227901 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Action: 'create', calling method: .version_select at 0x7f592af68820>, body: {"keypair": {"name": "tempest-keypair-1923133962", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBHQMdGCyMT1hbDwSzJbIC3vGw/50hh9pNWiOnw9OC3aZklDLPgxt7fsD+aIbPJw+1Hq/ZglvCNbJ65QeyY6uUYFifsBGDdQVXENRK79Jy0rAabG4kI+TIzMqTqMlpMOD5w=="}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:25.205341 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-70610fa9-5aae-4cff-a924-42a216227901 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Getting quotas for project c0e363b1584848c1aecbe186cf91eec5. Resources: {'key_pairs'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:45:25.209718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-70610fa9-5aae-4cff-a924-42a216227901 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Getting quotas for user 467992b53c4749858fd8b0ccd2db2393 and project c0e363b1584848c1aecbe186cf91eec5. Resources: {'key_pairs'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:45:25.230227 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-23e1f807-b6c1-44f2-a090-e6510d355c6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Flavor limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:45:25.230950 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-23e1f807-b6c1-44f2-a090-e6510d355c6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Image limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:45:25.231743 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-23e1f807-b6c1-44f2-a090-e6510d355c6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Flavor pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:45:25.232322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-23e1f807-b6c1-44f2-a090-e6510d355c6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Image pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:45:25.232902 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-23e1f807-b6c1-44f2-a090-e6510d355c6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:45:25.240625 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-70610fa9-5aae-4cff-a924-42a216227901 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.054496 Jul 11 14:45:25.241234 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 369/718] 10.222.0.72 () {60 vars in 1158 bytes} [Mon Jul 11 14:45:25 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:45:25.247016 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-23e1f807-b6c1-44f2-a090-e6510d355c6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:25.247809 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-23e1f807-b6c1-44f2-a090-e6510d355c6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:25.249261 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-23e1f807-b6c1-44f2-a090-e6510d355c6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:25.250050 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-23e1f807-b6c1-44f2-a090-e6510d355c6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:25.281108 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-23e1f807-b6c1-44f2-a090-e6510d355c6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Getting quotas for project 3d541b7a72dc4b4494a42bda6fa75026. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:45:25.288294 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-23e1f807-b6c1-44f2-a090-e6510d355c6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Getting quotas for user 0f8e300cd96e446db7430044e69d997a and project 3d541b7a72dc4b4494a42bda6fa75026. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:45:25.353681 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-23e1f807-b6c1-44f2-a090-e6510d355c6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Going to run 1 instances... {{(pid=130160) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:45:25.355976 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-23e1f807-b6c1-44f2-a090-e6510d355c6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='ccbe23b1-2e12-4e18-9982-7c804ff58b5f',volume_size=None,volume_type=None)] {{(pid=130160) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:45:25.508086 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-23e1f807-b6c1-44f2-a090-e6510d355c6f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.184498 Jul 11 14:45:25.508884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 350/719] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:45:24 2022] POST /compute/v2.1/servers => generated 406 bytes in 1186 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 11 14:45:25.520710 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1ac309e9-a4b1-423f-a283-a64e37475bd2 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:25.521428 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1ac309e9-a4b1-423f-a283-a64e37475bd2 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:25.595209 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1ac309e9-a4b1-423f-a283-a64e37475bd2 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:25.604566 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1ac309e9-a4b1-423f-a283-a64e37475bd2 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1095 microversion: 2.1 time: 0.087881 Jul 11 14:45:25.605172 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 370/720] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:25 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1095 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:26.552680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-4a07a581-d72a-4955-bf56-aeaab647e735 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1047303097", "imageRef": "a588fa24-23b5-4e53-87c6-7e600ee3a5ef", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "208c5447-9d23-4b47-b27a-4307a7296819"}], "security_groups": [{"name": "tempest-securitygroup--839624147"}], "key_name": "tempest-keypair-1923133962", "user_data": "}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:26.622130 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e138187c-a29f-4d64-a7d7-2624103bc87c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:26.623754 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e138187c-a29f-4d64-a7d7-2624103bc87c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:26.631361 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e138187c-a29f-4d64-a7d7-2624103bc87c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:26.632186 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e138187c-a29f-4d64-a7d7-2624103bc87c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:26.633420 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e138187c-a29f-4d64-a7d7-2624103bc87c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:26.634683 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e138187c-a29f-4d64-a7d7-2624103bc87c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:26.638614 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:26.667516 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.network.neutron [None req-4a07a581-d72a-4955-bf56-aeaab647e735 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] validate_networks() for [('208c5447-9d23-4b47-b27a-4307a7296819', None, None, None, None, None)] {{(pid=130160) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:45:26.712835 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e138187c-a29f-4d64-a7d7-2624103bc87c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:26.721535 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e138187c-a29f-4d64-a7d7-2624103bc87c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1193 microversion: 2.1 time: 0.103648 Jul 11 14:45:26.722371 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 371/721] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:26 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1193 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:45:27.310717 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-4a07a581-d72a-4955-bf56-aeaab647e735 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Flavor limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:45:27.311142 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-4a07a581-d72a-4955-bf56-aeaab647e735 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Image limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:45:27.311555 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-4a07a581-d72a-4955-bf56-aeaab647e735 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Flavor pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:45:27.311959 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-4a07a581-d72a-4955-bf56-aeaab647e735 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Image pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:45:27.312491 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-4a07a581-d72a-4955-bf56-aeaab647e735 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:45:27.323418 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4a07a581-d72a-4955-bf56-aeaab647e735 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:27.324282 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4a07a581-d72a-4955-bf56-aeaab647e735 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:27.325249 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4a07a581-d72a-4955-bf56-aeaab647e735 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:27.325779 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4a07a581-d72a-4955-bf56-aeaab647e735 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:27.347332 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-4a07a581-d72a-4955-bf56-aeaab647e735 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Getting quotas for project c0e363b1584848c1aecbe186cf91eec5. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:45:27.355315 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-4a07a581-d72a-4955-bf56-aeaab647e735 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Getting quotas for user 467992b53c4749858fd8b0ccd2db2393 and project c0e363b1584848c1aecbe186cf91eec5. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:45:27.372234 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-4a07a581-d72a-4955-bf56-aeaab647e735 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Going to run 1 instances... {{(pid=130160) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:45:27.375469 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-4a07a581-d72a-4955-bf56-aeaab647e735 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a588fa24-23b5-4e53-87c6-7e600ee3a5ef',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130160) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:45:27.432921 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-fcf8920e-9512-4174-8c0f-bb89409b0cd2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:27.436307 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-4a07a581-d72a-4955-bf56-aeaab647e735 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.888843 Jul 11 14:45:27.438633 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 351/722] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:45:26 2022] POST /compute/v2.1/servers => generated 401 bytes in 892 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 11 14:45:27.443738 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-fcf8920e-9512-4174-8c0f-bb89409b0cd2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:27.451210 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-785d1368-4dc7-4658-9686-9e469a82a268 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:27.453483 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-785d1368-4dc7-4658-9686-9e469a82a268 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:27.485413 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fcf8920e-9512-4174-8c0f-bb89409b0cd2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:27.485870 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fcf8920e-9512-4174-8c0f-bb89409b0cd2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:27.486722 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fcf8920e-9512-4174-8c0f-bb89409b0cd2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:27.487204 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fcf8920e-9512-4174-8c0f-bb89409b0cd2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:27.530270 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-fcf8920e-9512-4174-8c0f-bb89409b0cd2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/ips" status: 200 len: 150 microversion: 2.1 time: 0.272487 Jul 11 14:45:27.531107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 372/723] 10.222.0.72 () {58 vars in 1247 bytes} [Mon Jul 11 14:45:27 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/ips => generated 150 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:45:27.571763 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-785d1368-4dc7-4658-9686-9e469a82a268 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:27.587608 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-785d1368-4dc7-4658-9686-9e469a82a268 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1238 microversion: 2.1 time: 0.144262 Jul 11 14:45:27.595043 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 352/724] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:27 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1238 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:27.756082 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d89a17de-f753-48e9-873f-3214bbe2b4e6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:27.757217 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d89a17de-f753-48e9-873f-3214bbe2b4e6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:27.791284 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d89a17de-f753-48e9-873f-3214bbe2b4e6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:27.791693 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d89a17de-f753-48e9-873f-3214bbe2b4e6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:27.792497 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d89a17de-f753-48e9-873f-3214bbe2b4e6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:27.792939 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d89a17de-f753-48e9-873f-3214bbe2b4e6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:27.954104 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d89a17de-f753-48e9-873f-3214bbe2b4e6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:27.962768 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-d89a17de-f753-48e9-873f-3214bbe2b4e6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1267 microversion: 2.1 time: 0.224289 Jul 11 14:45:27.963468 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 373/725] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:27 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1267 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:28.607207 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-4b58ca4d-02e1-4623-9943-a84048ee2f6a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:28.608428 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-4b58ca4d-02e1-4623-9943-a84048ee2f6a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:28.616630 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4b58ca4d-02e1-4623-9943-a84048ee2f6a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:28.617339 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4b58ca4d-02e1-4623-9943-a84048ee2f6a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:28.618143 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4b58ca4d-02e1-4623-9943-a84048ee2f6a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:28.618616 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4b58ca4d-02e1-4623-9943-a84048ee2f6a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:28.704126 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4b58ca4d-02e1-4623-9943-a84048ee2f6a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:28.713323 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-4b58ca4d-02e1-4623-9943-a84048ee2f6a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1290 microversion: 2.1 time: 0.111989 Jul 11 14:45:28.714232 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 353/726] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:28 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1290 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:28.905839 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-87c157dc-79af-4664-a401-4733aeee2c38 req-beb13f63-2ed4-4a60-b75d-caea1a308d55 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1009499c-a8c8-472c-bef5-fdf42c002c0b", "tag": "1544341d-a8ad-41b0-8bc7-e3d71545d58f"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:28.919598 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-87c157dc-79af-4664-a401-4733aeee2c38 req-beb13f63-2ed4-4a60-b75d-caea1a308d55 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:28.920674 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-87c157dc-79af-4664-a401-4733aeee2c38 req-beb13f63-2ed4-4a60-b75d-caea1a308d55 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:28.939830 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-87c157dc-79af-4664-a401-4733aeee2c38 req-beb13f63-2ed4-4a60-b75d-caea1a308d55 service nova] Creating event network-changed:1544341d-a8ad-41b0-8bc7-e3d71545d58f for instance 1009499c-a8c8-472c-bef5-fdf42c002c0b on hv-cinder-maste Jul 11 14:45:28.947679 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-87c157dc-79af-4664-a401-4733aeee2c38 req-beb13f63-2ed4-4a60-b75d-caea1a308d55 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045444 Jul 11 14:45:28.951159 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 374/727] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:45:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:45:28.981229 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a94d5ddf-8298-4958-b746-780e835ed4bb tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:28.981852 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a94d5ddf-8298-4958-b746-780e835ed4bb tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:28.988807 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a94d5ddf-8298-4958-b746-780e835ed4bb tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:28.989538 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a94d5ddf-8298-4958-b746-780e835ed4bb tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:28.991137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a94d5ddf-8298-4958-b746-780e835ed4bb tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:28.992246 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a94d5ddf-8298-4958-b746-780e835ed4bb tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:29.054302 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-56d48759-c98a-4bef-943c-2c8d5c50e0dc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "99108f16-9c07-4564-a36d-0c758fff0b33"}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:29.056095 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-56d48759-c98a-4bef-943c-2c8d5c50e0dc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:29.064366 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-56d48759-c98a-4bef-943c-2c8d5c50e0dc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:29.065240 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-56d48759-c98a-4bef-943c-2c8d5c50e0dc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:29.066982 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-56d48759-c98a-4bef-943c-2c8d5c50e0dc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:29.067653 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-56d48759-c98a-4bef-943c-2c8d5c50e0dc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:29.241518 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a94d5ddf-8298-4958-b746-780e835ed4bb tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:29.254376 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a94d5ddf-8298-4958-b746-780e835ed4bb tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1267 microversion: 2.1 time: 0.277228 Jul 11 14:45:29.255845 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 354/728] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:28 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1267 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:29.399600 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-56d48759-c98a-4bef-943c-2c8d5c50e0dc tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "POST /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.350130 Jul 11 14:45:29.400547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 375/729] 10.222.0.72 () {60 vars in 1322 bytes} [Mon Jul 11 14:45:29 2022] POST /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments => generated 194 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:45:29.731122 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-98652bb5-e7c7-4ca8-a538-00793ebcb493 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:29.731835 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-98652bb5-e7c7-4ca8-a538-00793ebcb493 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:29.743929 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-98652bb5-e7c7-4ca8-a538-00793ebcb493 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:29.744525 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-98652bb5-e7c7-4ca8-a538-00793ebcb493 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:29.745311 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-98652bb5-e7c7-4ca8-a538-00793ebcb493 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:29.745750 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-98652bb5-e7c7-4ca8-a538-00793ebcb493 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:29.881460 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-98652bb5-e7c7-4ca8-a538-00793ebcb493 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:29.890281 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-98652bb5-e7c7-4ca8-a538-00793ebcb493 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1363 microversion: 2.1 time: 0.163071 Jul 11 14:45:29.890912 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 355/730] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:29 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1363 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:30.271897 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-ebc7ef3a-e07f-4b5a-8ed9-f2f5c84a0d9f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:30.273313 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-ebc7ef3a-e07f-4b5a-8ed9-f2f5c84a0d9f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:30.280266 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ebc7ef3a-e07f-4b5a-8ed9-f2f5c84a0d9f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:30.280728 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ebc7ef3a-e07f-4b5a-8ed9-f2f5c84a0d9f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:30.281541 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ebc7ef3a-e07f-4b5a-8ed9-f2f5c84a0d9f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:30.282179 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ebc7ef3a-e07f-4b5a-8ed9-f2f5c84a0d9f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:30.513029 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-ebc7ef3a-e07f-4b5a-8ed9-f2f5c84a0d9f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:30.524156 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-ebc7ef3a-e07f-4b5a-8ed9-f2f5c84a0d9f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1267 microversion: 2.1 time: 0.257018 Jul 11 14:45:30.524746 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 376/731] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:30 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1267 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:30.844009 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-d2bcbe4d-0dbf-4be1-bd96-4af5e7261a3e req-11049f85-a6e8-4991-84a2-75ee8c97cf08 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cb814dfe-6200-40ed-832f-df4face7a16e", "tag": "acddbdf4-6edc-4eaf-8287-a34936c1b912"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:30.857029 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-d2bcbe4d-0dbf-4be1-bd96-4af5e7261a3e req-11049f85-a6e8-4991-84a2-75ee8c97cf08 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:30.857475 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-d2bcbe4d-0dbf-4be1-bd96-4af5e7261a3e req-11049f85-a6e8-4991-84a2-75ee8c97cf08 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:30.869647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-d2bcbe4d-0dbf-4be1-bd96-4af5e7261a3e req-11049f85-a6e8-4991-84a2-75ee8c97cf08 service nova] Creating event network-changed:acddbdf4-6edc-4eaf-8287-a34936c1b912 for instance cb814dfe-6200-40ed-832f-df4face7a16e on hv-cinder-maste Jul 11 14:45:30.874918 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-d2bcbe4d-0dbf-4be1-bd96-4af5e7261a3e req-11049f85-a6e8-4991-84a2-75ee8c97cf08 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034903 Jul 11 14:45:30.875960 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 356/732] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:45:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:45:30.908429 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c078c4cd-f651-4e83-bad0-e6b9e3b0b4a5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:30.909148 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c078c4cd-f651-4e83-bad0-e6b9e3b0b4a5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:30.916400 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c078c4cd-f651-4e83-bad0-e6b9e3b0b4a5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:30.916852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c078c4cd-f651-4e83-bad0-e6b9e3b0b4a5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:30.917630 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c078c4cd-f651-4e83-bad0-e6b9e3b0b4a5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:30.918068 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c078c4cd-f651-4e83-bad0-e6b9e3b0b4a5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:31.137819 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c078c4cd-f651-4e83-bad0-e6b9e3b0b4a5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:31.145591 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c078c4cd-f651-4e83-bad0-e6b9e3b0b4a5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1363 microversion: 2.1 time: 0.241286 Jul 11 14:45:31.146925 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 377/733] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:30 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1363 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:31.541801 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-cc7f9b49-d6c1-497b-a41c-052baf84e791 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:31.542548 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-cc7f9b49-d6c1-497b-a41c-052baf84e791 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:31.549432 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc7f9b49-d6c1-497b-a41c-052baf84e791 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:31.549890 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc7f9b49-d6c1-497b-a41c-052baf84e791 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:31.550907 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc7f9b49-d6c1-497b-a41c-052baf84e791 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:31.551345 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc7f9b49-d6c1-497b-a41c-052baf84e791 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:31.683043 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-cc7f9b49-d6c1-497b-a41c-052baf84e791 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:31.685367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:31.692100 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-cc7f9b49-d6c1-497b-a41c-052baf84e791 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1267 microversion: 2.1 time: 0.154559 Jul 11 14:45:31.696464 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 357/734] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:31 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1267 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:32.218632 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9948ef6c-ff67-4c74-94db-b9bbd84fcca3 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:32.219315 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9948ef6c-ff67-4c74-94db-b9bbd84fcca3 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:32.226677 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9948ef6c-ff67-4c74-94db-b9bbd84fcca3 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:32.227282 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9948ef6c-ff67-4c74-94db-b9bbd84fcca3 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:32.228525 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9948ef6c-ff67-4c74-94db-b9bbd84fcca3 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:32.228988 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9948ef6c-ff67-4c74-94db-b9bbd84fcca3 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:32.403419 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9948ef6c-ff67-4c74-94db-b9bbd84fcca3 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:32.413009 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9948ef6c-ff67-4c74-94db-b9bbd84fcca3 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1363 microversion: 2.1 time: 0.253350 Jul 11 14:45:32.413602 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 378/735] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:32 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1363 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:32.711061 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-fc379ee4-6464-475f-ab52-824003ada494 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:32.712138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-fc379ee4-6464-475f-ab52-824003ada494 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:32.722853 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fc379ee4-6464-475f-ab52-824003ada494 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:32.723401 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fc379ee4-6464-475f-ab52-824003ada494 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:32.724437 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fc379ee4-6464-475f-ab52-824003ada494 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:32.725062 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fc379ee4-6464-475f-ab52-824003ada494 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:32.944607 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-fc379ee4-6464-475f-ab52-824003ada494 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:32.952870 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-fc379ee4-6464-475f-ab52-824003ada494 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1267 microversion: 2.1 time: 0.247427 Jul 11 14:45:32.953769 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 358/736] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:32 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1267 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:33.431543 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-5c461698-254e-4c85-924f-9d24f8479f2b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:33.432121 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-5c461698-254e-4c85-924f-9d24f8479f2b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:33.435274 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=88,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:33.442503 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5c461698-254e-4c85-924f-9d24f8479f2b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:33.443681 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5c461698-254e-4c85-924f-9d24f8479f2b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:33.444611 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5c461698-254e-4c85-924f-9d24f8479f2b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:33.445178 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5c461698-254e-4c85-924f-9d24f8479f2b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:33.592283 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-5c461698-254e-4c85-924f-9d24f8479f2b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:33.600874 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-5c461698-254e-4c85-924f-9d24f8479f2b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1363 microversion: 2.1 time: 0.173915 Jul 11 14:45:33.601464 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 379/737] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:33 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1363 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:33.977626 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-de39dae1-eae6-4040-b06b-0a2864b3acda tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:33.978387 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-de39dae1-eae6-4040-b06b-0a2864b3acda tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:33.985199 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-de39dae1-eae6-4040-b06b-0a2864b3acda tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:33.985649 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-de39dae1-eae6-4040-b06b-0a2864b3acda tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:33.986633 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-de39dae1-eae6-4040-b06b-0a2864b3acda tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:33.986892 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-de39dae1-eae6-4040-b06b-0a2864b3acda tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:34.159352 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-de39dae1-eae6-4040-b06b-0a2864b3acda tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:34.167600 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-de39dae1-eae6-4040-b06b-0a2864b3acda tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1267 microversion: 2.1 time: 0.194407 Jul 11 14:45:34.168182 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 359/738] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:33 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1267 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:34.628778 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-767d4c57-17bb-40b6-9730-87d2e24e39d4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:34.629838 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-767d4c57-17bb-40b6-9730-87d2e24e39d4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:34.639237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-767d4c57-17bb-40b6-9730-87d2e24e39d4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:34.639993 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-767d4c57-17bb-40b6-9730-87d2e24e39d4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:34.641319 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-767d4c57-17bb-40b6-9730-87d2e24e39d4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:34.642109 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-767d4c57-17bb-40b6-9730-87d2e24e39d4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:34.794258 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d7ee27fc-5e07-4934-b0db-2d5ecbf4141e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:34.795131 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d7ee27fc-5e07-4934-b0db-2d5ecbf4141e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:34.805459 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d7ee27fc-5e07-4934-b0db-2d5ecbf4141e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:34.807192 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d7ee27fc-5e07-4934-b0db-2d5ecbf4141e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:34.808765 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d7ee27fc-5e07-4934-b0db-2d5ecbf4141e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:34.809739 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d7ee27fc-5e07-4934-b0db-2d5ecbf4141e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:34.849366 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-767d4c57-17bb-40b6-9730-87d2e24e39d4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:34.859215 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-767d4c57-17bb-40b6-9730-87d2e24e39d4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1363 microversion: 2.1 time: 0.237042 Jul 11 14:45:34.860682 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 380/739] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:34 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1363 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:34.925909 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.objects.instance [None req-d7ee27fc-5e07-4934-b0db-2d5ecbf4141e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lazy-loading 'services' on Instance uuid f66823cd-4fe8-42cf-ab7a-d7cd7afc289a {{(pid=130160) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:45:35.015728 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d7ee27fc-5e07-4934-b0db-2d5ecbf4141e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "DELETE /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-volume_attachments/2be6be9e-10da-4656-a426-f2a29458eb3c" status: 202 len: 0 microversion: 2.1 time: 0.226824 Jul 11 14:45:35.017270 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 360/740] 10.222.0.72 () {58 vars in 1415 bytes} [Mon Jul 11 14:45:34 2022] DELETE /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-volume_attachments/2be6be9e-10da-4656-a426-f2a29458eb3c => generated 0 bytes in 230 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 11 14:45:35.025901 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c3b7c9cb-df90-401a-a9fe-3695ce086f12 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:35.026936 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c3b7c9cb-df90-401a-a9fe-3695ce086f12 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:35.033210 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c3b7c9cb-df90-401a-a9fe-3695ce086f12 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:35.033658 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c3b7c9cb-df90-401a-a9fe-3695ce086f12 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:35.034473 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c3b7c9cb-df90-401a-a9fe-3695ce086f12 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:35.034899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c3b7c9cb-df90-401a-a9fe-3695ce086f12 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:35.070754 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c3b7c9cb-df90-401a-a9fe-3695ce086f12 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048379 Jul 11 14:45:35.071794 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 381/741] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:45:35 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:45:35.185348 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7ee9eea9-7056-4239-abdc-2e95d1a39586 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:35.185940 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-7ee9eea9-7056-4239-abdc-2e95d1a39586 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:35.193557 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7ee9eea9-7056-4239-abdc-2e95d1a39586 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:35.194005 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7ee9eea9-7056-4239-abdc-2e95d1a39586 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:35.194828 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7ee9eea9-7056-4239-abdc-2e95d1a39586 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:35.195267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7ee9eea9-7056-4239-abdc-2e95d1a39586 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:35.386392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7ee9eea9-7056-4239-abdc-2e95d1a39586 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:35.396121 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-7ee9eea9-7056-4239-abdc-2e95d1a39586 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1267 microversion: 2.1 time: 0.214859 Jul 11 14:45:35.397174 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 361/742] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:35 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1267 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:35.879413 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=6 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:35.885831 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d359c27f-e2b2-4fa1-ab17-57aa4792827f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:35.886686 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d359c27f-e2b2-4fa1-ab17-57aa4792827f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:35.896458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d359c27f-e2b2-4fa1-ab17-57aa4792827f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:35.897200 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d359c27f-e2b2-4fa1-ab17-57aa4792827f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:35.898718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d359c27f-e2b2-4fa1-ab17-57aa4792827f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:35.899437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d359c27f-e2b2-4fa1-ab17-57aa4792827f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:36.053915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-5bfeebfe-02f7-4af0-a32b-b3898d9d817c req-51dbe1f9-e5b0-47c4-8053-e84fcd20191a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb814dfe-6200-40ed-832f-df4face7a16e", "name": "network-vif-plugged", "status": "completed", "tag": "acddbdf4-6edc-4eaf-8287-a34936c1b912"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:36.072847 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-5bfeebfe-02f7-4af0-a32b-b3898d9d817c req-51dbe1f9-e5b0-47c4-8053-e84fcd20191a service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:36.073640 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-5bfeebfe-02f7-4af0-a32b-b3898d9d817c req-51dbe1f9-e5b0-47c4-8053-e84fcd20191a service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:36.077626 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d359c27f-e2b2-4fa1-ab17-57aa4792827f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:36.093923 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-5bfeebfe-02f7-4af0-a32b-b3898d9d817c req-51dbe1f9-e5b0-47c4-8053-e84fcd20191a service nova] Creating event network-vif-plugged:acddbdf4-6edc-4eaf-8287-a34936c1b912 for instance cb814dfe-6200-40ed-832f-df4face7a16e on hv-cinder-maste Jul 11 14:45:36.097451 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-d359c27f-e2b2-4fa1-ab17-57aa4792827f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1363 microversion: 2.1 time: 0.218906 Jul 11 14:45:36.098793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 382/743] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:35 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1363 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:36.100424 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-5bfeebfe-02f7-4af0-a32b-b3898d9d817c req-51dbe1f9-e5b0-47c4-8053-e84fcd20191a service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.054113 Jul 11 14:45:36.101293 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 362/744] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:45:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:45:36.111299 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f6778050-a8eb-41fd-9420-ce38ca1e4737 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:36.112937 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f6778050-a8eb-41fd-9420-ce38ca1e4737 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:36.123943 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f6778050-a8eb-41fd-9420-ce38ca1e4737 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:36.124775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f6778050-a8eb-41fd-9420-ce38ca1e4737 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:36.126390 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f6778050-a8eb-41fd-9420-ce38ca1e4737 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:36.127211 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f6778050-a8eb-41fd-9420-ce38ca1e4737 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:36.171480 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f6778050-a8eb-41fd-9420-ce38ca1e4737 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069338 Jul 11 14:45:36.172060 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 383/745] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:45:36 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:45:36.416621 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-78f6b9f8-c7bf-4c98-8140-d124895747aa tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:36.417352 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-78f6b9f8-c7bf-4c98-8140-d124895747aa tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:36.423832 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-78f6b9f8-c7bf-4c98-8140-d124895747aa tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:36.424274 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-78f6b9f8-c7bf-4c98-8140-d124895747aa tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:36.425042 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-78f6b9f8-c7bf-4c98-8140-d124895747aa tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:36.425475 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-78f6b9f8-c7bf-4c98-8140-d124895747aa tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:36.606256 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-78f6b9f8-c7bf-4c98-8140-d124895747aa tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:36.615851 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-78f6b9f8-c7bf-4c98-8140-d124895747aa tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1267 microversion: 2.1 time: 0.203341 Jul 11 14:45:36.616451 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 363/746] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:36 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1267 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:37.118664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1a302089-7e25-4577-b590-65bfd1e2fcb2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:37.119290 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1a302089-7e25-4577-b590-65bfd1e2fcb2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:37.126083 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1a302089-7e25-4577-b590-65bfd1e2fcb2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:37.126578 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1a302089-7e25-4577-b590-65bfd1e2fcb2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:37.127357 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1a302089-7e25-4577-b590-65bfd1e2fcb2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:37.127796 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1a302089-7e25-4577-b590-65bfd1e2fcb2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:37.184062 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-25401bff-f56c-4c53-ab8b-e53f1e05e39e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:37.184896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-25401bff-f56c-4c53-ab8b-e53f1e05e39e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:37.191500 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-25401bff-f56c-4c53-ab8b-e53f1e05e39e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:37.191950 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-25401bff-f56c-4c53-ab8b-e53f1e05e39e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:37.193134 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-25401bff-f56c-4c53-ab8b-e53f1e05e39e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:37.193572 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-25401bff-f56c-4c53-ab8b-e53f1e05e39e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:37.226724 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-25401bff-f56c-4c53-ab8b-e53f1e05e39e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046616 Jul 11 14:45:37.227323 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 364/747] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:45:37 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:45:37.299219 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1a302089-7e25-4577-b590-65bfd1e2fcb2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:37.307230 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1a302089-7e25-4577-b590-65bfd1e2fcb2 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1363 microversion: 2.1 time: 0.193022 Jul 11 14:45:37.307829 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 384/748] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:37 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1363 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:45:37.634057 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-caff4417-8005-4748-9d0b-5a45bf869892 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:37.634714 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-caff4417-8005-4748-9d0b-5a45bf869892 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:37.636752 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=4 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:37.637439 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=14 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:37.644046 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-caff4417-8005-4748-9d0b-5a45bf869892 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:37.644499 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-caff4417-8005-4748-9d0b-5a45bf869892 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:37.645280 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-caff4417-8005-4748-9d0b-5a45bf869892 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:37.646330 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-caff4417-8005-4748-9d0b-5a45bf869892 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:37.819560 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-caff4417-8005-4748-9d0b-5a45bf869892 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:37.830404 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-caff4417-8005-4748-9d0b-5a45bf869892 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1267 microversion: 2.1 time: 0.200924 Jul 11 14:45:37.831026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 365/749] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:37 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1267 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:38.159879 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-4e3e109f-bb05-418d-a30a-3be867a072e3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "99108f16-9c07-4564-a36d-0c758fff0b33"}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:38.161943 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-4e3e109f-bb05-418d-a30a-3be867a072e3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:38.173434 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4e3e109f-bb05-418d-a30a-3be867a072e3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:38.174071 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4e3e109f-bb05-418d-a30a-3be867a072e3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:38.175065 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4e3e109f-bb05-418d-a30a-3be867a072e3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:38.175703 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4e3e109f-bb05-418d-a30a-3be867a072e3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:38.238062 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2e87b77b-2ed8-402f-89c3-aae82e0246e4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:38.239890 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2e87b77b-2ed8-402f-89c3-aae82e0246e4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:38.242308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=6 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:38.253585 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2e87b77b-2ed8-402f-89c3-aae82e0246e4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:38.254051 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2e87b77b-2ed8-402f-89c3-aae82e0246e4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:38.254869 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2e87b77b-2ed8-402f-89c3-aae82e0246e4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:38.255890 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2e87b77b-2ed8-402f-89c3-aae82e0246e4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:38.285829 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-4e3e109f-bb05-418d-a30a-3be867a072e3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] HTTP exception thrown: Invalid volume: volume 99108f16-9c07-4564-a36d-0c758fff0b33 is already attached to instances: 1f161996-6f14-46a9-9200-c8763cb67e21 Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-4e3e109f-bb05-418d-a30a-3be867a072e3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Request method failure captured: Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: POST /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments HTTP/1.1 Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 74 Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZbQUJdl7M18phk4FM9-GGzi8XbGx24YhEK9KsOfI165n5v27J9wq7wzJjs1m7qkLCobj_5JRaSFQlNcVRL5jfwSqdoh1nodKIJxEOpM9gSJP97YpvVDnXLPsMNjwZQwG04QTf0uG5zLXZ7eKV_ivBXCJkCsdmhsPnE9ln-9M7wHs4 Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:45:38.287661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:45:38.289563 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}] Jul 11 14:45:38.289563 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:45:38.289563 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:45:38.289563 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:45:38.289563 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:45:38.289563 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:45:38.289563 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:45:38.289563 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 3ae56fd1172f4bf8806eefc6e8877fef Jul 11 14:45:38.289563 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:45:38.289563 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Jul 11 14:45:38.289563 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {"volumeAttachment": {"volumeId": "99108f16-9c07-4564-a36d-0c758fff0b33"}} Jul 11 14:45:38.289563 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Invalid volume: volume 99108f16-9c07-4564-a36d-0c758fff0b33 is already attached to instances: 1f161996-6f14-46a9-9200-c8763cb67e21 Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'server_id': '1f161996-6f14-46a9-9200-c8763cb67e21', 'body': {'volumeAttachment': {'volumeId': '99108f16-9c07-4564-a36d-0c758fff0b33'}}} Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 238, in inner Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 164, in inner Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4985, in attach_volume Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4825, in _check_volume_already_attached Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 99108f16-9c07-4564-a36d-0c758fff0b33 is already attached to instances: 1f161996-6f14-46a9-9200-c8763cb67e21 Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:38.291325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 99108f16-9c07-4564-a36d-0c758fff0b33 is already attached to instances: 1f161996-6f14-46a9-9200-c8763cb67e21 Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:38.292852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:45:38.295131 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:45:38.295131 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:45:38.295131 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:45:38.295131 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 99108f16-9c07-4564-a36d-0c758fff0b33 is already attached to instances: 1f161996-6f14-46a9-9200-c8763cb67e21 Jul 11 14:45:38.295131 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:38.295131 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-4e3e109f-bb05-418d-a30a-3be867a072e3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Returning 400 to user: Invalid volume: volume 99108f16-9c07-4564-a36d-0c758fff0b33 is already attached to instances: 1f161996-6f14-46a9-9200-c8763cb67e21 {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:45:38.295131 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-4e3e109f-bb05-418d-a30a-3be867a072e3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "POST /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.137006 Jul 11 14:45:38.295131 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 385/750] 10.222.0.72 () {60 vars in 1322 bytes} [Mon Jul 11 14:45:38 2022] POST /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments => generated 174 bytes in 138 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 11 14:45:38.303942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2e87b77b-2ed8-402f-89c3-aae82e0246e4 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069835 Jul 11 14:45:38.305367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 366/751] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:45:38 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:45:38.326277 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a2f3336d-36bd-4e00-919a-941109e2e852 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:38.327022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a2f3336d-36bd-4e00-919a-941109e2e852 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:38.335636 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a2f3336d-36bd-4e00-919a-941109e2e852 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:38.336242 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a2f3336d-36bd-4e00-919a-941109e2e852 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:38.337223 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a2f3336d-36bd-4e00-919a-941109e2e852 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:38.337474 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a2f3336d-36bd-4e00-919a-941109e2e852 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:38.387022 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-27745c68-7702-44b6-b220-373bf7c964dd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:38.387915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-27745c68-7702-44b6-b220-373bf7c964dd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:38.398940 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-27745c68-7702-44b6-b220-373bf7c964dd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:38.399670 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-27745c68-7702-44b6-b220-373bf7c964dd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:38.400815 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-27745c68-7702-44b6-b220-373bf7c964dd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:38.401256 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-27745c68-7702-44b6-b220-373bf7c964dd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:38.516833 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.objects.instance [None req-27745c68-7702-44b6-b220-373bf7c964dd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lazy-loading 'services' on Instance uuid 1f161996-6f14-46a9-9200-c8763cb67e21 {{(pid=130160) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:45:38.522026 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a2f3336d-36bd-4e00-919a-941109e2e852 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:38.539217 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a2f3336d-36bd-4e00-919a-941109e2e852 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1363 microversion: 2.1 time: 0.216785 Jul 11 14:45:38.540004 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 386/752] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:38 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1363 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:38.606608 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-27745c68-7702-44b6-b220-373bf7c964dd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "DELETE /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments/99108f16-9c07-4564-a36d-0c758fff0b33" status: 202 len: 0 microversion: 2.1 time: 0.227114 Jul 11 14:45:38.607444 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 367/753] 10.222.0.72 () {58 vars in 1415 bytes} [Mon Jul 11 14:45:38 2022] DELETE /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments/99108f16-9c07-4564-a36d-0c758fff0b33 => generated 0 bytes in 230 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:45:38.616741 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8d7a0179-c71e-48d4-acb6-76ffb541daf6 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:38.619973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8d7a0179-c71e-48d4-acb6-76ffb541daf6 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:38.626299 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8d7a0179-c71e-48d4-acb6-76ffb541daf6 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:38.626781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8d7a0179-c71e-48d4-acb6-76ffb541daf6 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:38.627562 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8d7a0179-c71e-48d4-acb6-76ffb541daf6 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:38.627994 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8d7a0179-c71e-48d4-acb6-76ffb541daf6 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:38.650599 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:38.669824 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8d7a0179-c71e-48d4-acb6-76ffb541daf6 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056044 Jul 11 14:45:38.670730 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 387/754] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:45:38 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:45:38.850817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d1c80810-f5c6-441b-ac64-e25690ccfa4f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:38.851661 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d1c80810-f5c6-441b-ac64-e25690ccfa4f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:38.859149 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d1c80810-f5c6-441b-ac64-e25690ccfa4f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:38.859608 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d1c80810-f5c6-441b-ac64-e25690ccfa4f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:38.860398 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d1c80810-f5c6-441b-ac64-e25690ccfa4f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:38.860839 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d1c80810-f5c6-441b-ac64-e25690ccfa4f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:39.024257 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d1c80810-f5c6-441b-ac64-e25690ccfa4f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:39.032263 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d1c80810-f5c6-441b-ac64-e25690ccfa4f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1267 microversion: 2.1 time: 0.187407 Jul 11 14:45:39.032838 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 368/755] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:38 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1267 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:39.322277 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-ebf03fa3-f123-4925-84e3-56837aab5634 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:39.325022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-ebf03fa3-f123-4925-84e3-56837aab5634 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:39.339658 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ebf03fa3-f123-4925-84e3-56837aab5634 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:39.340512 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ebf03fa3-f123-4925-84e3-56837aab5634 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:39.342168 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ebf03fa3-f123-4925-84e3-56837aab5634 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:39.342971 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ebf03fa3-f123-4925-84e3-56837aab5634 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:39.376168 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-ebf03fa3-f123-4925-84e3-56837aab5634 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.062654 Jul 11 14:45:39.376953 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 388/756] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:45:39 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a/os-volume_attachments => generated 25 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 11 14:45:39.479875 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3183d59a-8865-4ade-8264-2d4521f59678 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:39.480787 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-3183d59a-8865-4ade-8264-2d4521f59678 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:39.493416 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3183d59a-8865-4ade-8264-2d4521f59678 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:39.494234 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3183d59a-8865-4ade-8264-2d4521f59678 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:39.495635 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3183d59a-8865-4ade-8264-2d4521f59678 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:39.496431 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3183d59a-8865-4ade-8264-2d4521f59678 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:39.528969 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-3183d59a-8865-4ade-8264-2d4521f59678 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Going to try to terminate instance {{(pid=130160) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:45:39.557759 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d6a696a9-eff8-43de-8caa-36be41bce767 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:39.558381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d6a696a9-eff8-43de-8caa-36be41bce767 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:39.566107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d6a696a9-eff8-43de-8caa-36be41bce767 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:39.567051 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d6a696a9-eff8-43de-8caa-36be41bce767 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:39.568091 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d6a696a9-eff8-43de-8caa-36be41bce767 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:39.569367 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d6a696a9-eff8-43de-8caa-36be41bce767 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:39.628842 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-3183d59a-8865-4ade-8264-2d4521f59678 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "DELETE /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 204 len: 0 microversion: 2.1 time: 0.156275 Jul 11 14:45:39.629613 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 369/757] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:45:39 2022] DELETE /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 11 14:45:39.641860 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d529ed98-a3d2-4b05-9559-dffc9776e822 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:39.642850 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d529ed98-a3d2-4b05-9559-dffc9776e822 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:39.651313 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d529ed98-a3d2-4b05-9559-dffc9776e822 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:39.651811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d529ed98-a3d2-4b05-9559-dffc9776e822 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:39.652728 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d529ed98-a3d2-4b05-9559-dffc9776e822 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:39.653289 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d529ed98-a3d2-4b05-9559-dffc9776e822 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:39.760217 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d6a696a9-eff8-43de-8caa-36be41bce767 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:39.771214 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-d6a696a9-eff8-43de-8caa-36be41bce767 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1550 microversion: 2.1 time: 0.218298 Jul 11 14:45:39.771800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 389/758] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:39 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1550 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:45:39.777048 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c744999c-1ec8-467d-b877-d9d6503979fd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:39.778193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c744999c-1ec8-467d-b877-d9d6503979fd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:39.790644 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c744999c-1ec8-467d-b877-d9d6503979fd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:39.791497 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c744999c-1ec8-467d-b877-d9d6503979fd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:39.792895 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c744999c-1ec8-467d-b877-d9d6503979fd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:39.793676 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c744999c-1ec8-467d-b877-d9d6503979fd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:39.854162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c744999c-1ec8-467d-b877-d9d6503979fd tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.080605 Jul 11 14:45:39.855265 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 390/759] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:45:39 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments => generated 197 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:45:39.864114 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-17087d25-18b7-4a6e-ab39-3a5569122b2a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:39.865601 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-17087d25-18b7-4a6e-ab39-3a5569122b2a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:39.873583 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d529ed98-a3d2-4b05-9559-dffc9776e822 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:39.876856 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-17087d25-18b7-4a6e-ab39-3a5569122b2a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:39.877491 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-17087d25-18b7-4a6e-ab39-3a5569122b2a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:39.880097 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-17087d25-18b7-4a6e-ab39-3a5569122b2a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:39.880917 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-17087d25-18b7-4a6e-ab39-3a5569122b2a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:39.885733 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d529ed98-a3d2-4b05-9559-dffc9776e822 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1685 microversion: 2.1 time: 0.249007 Jul 11 14:45:39.890503 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 370/760] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:39 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1685 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:45:39.997669 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-17087d25-18b7-4a6e-ab39-3a5569122b2a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.140770 Jul 11 14:45:39.998270 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 391/761] 10.222.0.72 () {58 vars in 1274 bytes} [Mon Jul 11 14:45:39 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-interface => generated 285 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:45:40.039627 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-263dd382-7030-4657-b331-03740602ee75 req-e4505180-8e3f-4bf0-bd0e-bcf5cc9f7c7a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1009499c-a8c8-472c-bef5-fdf42c002c0b", "name": "network-vif-plugged", "status": "completed", "tag": "1544341d-a8ad-41b0-8bc7-e3d71545d58f"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:40.053241 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-263dd382-7030-4657-b331-03740602ee75 req-e4505180-8e3f-4bf0-bd0e-bcf5cc9f7c7a service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:40.053688 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-263dd382-7030-4657-b331-03740602ee75 req-e4505180-8e3f-4bf0-bd0e-bcf5cc9f7c7a service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:40.068877 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-263dd382-7030-4657-b331-03740602ee75 req-e4505180-8e3f-4bf0-bd0e-bcf5cc9f7c7a service nova] Creating event network-vif-plugged:1544341d-a8ad-41b0-8bc7-e3d71545d58f for instance 1009499c-a8c8-472c-bef5-fdf42c002c0b on hv-cinder-maste Jul 11 14:45:40.070156 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e38bed2f-41fb-4582-9307-0ee29bbcaef0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:40.072106 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e38bed2f-41fb-4582-9307-0ee29bbcaef0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:40.075110 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-263dd382-7030-4657-b331-03740602ee75 req-e4505180-8e3f-4bf0-bd0e-bcf5cc9f7c7a service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040842 Jul 11 14:45:40.076258 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 371/762] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:45:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:45:40.087594 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e38bed2f-41fb-4582-9307-0ee29bbcaef0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:40.088880 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e38bed2f-41fb-4582-9307-0ee29bbcaef0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:40.089774 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e38bed2f-41fb-4582-9307-0ee29bbcaef0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:40.090231 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e38bed2f-41fb-4582-9307-0ee29bbcaef0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:40.268969 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e38bed2f-41fb-4582-9307-0ee29bbcaef0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:40.283809 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e38bed2f-41fb-4582-9307-0ee29bbcaef0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1267 microversion: 2.1 time: 0.227705 Jul 11 14:45:40.284867 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 392/763] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:40 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1267 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:40.871724 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e5e32f59-16e1-486a-b2ef-9882f3d08aa3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:40.873528 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-e5e32f59-16e1-486a-b2ef-9882f3d08aa3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:40.882654 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e5e32f59-16e1-486a-b2ef-9882f3d08aa3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:40.883290 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e5e32f59-16e1-486a-b2ef-9882f3d08aa3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:40.884259 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e5e32f59-16e1-486a-b2ef-9882f3d08aa3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:40.884811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e5e32f59-16e1-486a-b2ef-9882f3d08aa3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:40.908207 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-4fbdf5ce-be8c-45c3-b7f4-df4e63923436 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:40.909473 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-4fbdf5ce-be8c-45c3-b7f4-df4e63923436 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:40.915173 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=3,nova_cell1:UPDATE=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:40.919144 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4fbdf5ce-be8c-45c3-b7f4-df4e63923436 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:40.920009 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4fbdf5ce-be8c-45c3-b7f4-df4e63923436 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:40.921311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4fbdf5ce-be8c-45c3-b7f4-df4e63923436 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:40.922019 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4fbdf5ce-be8c-45c3-b7f4-df4e63923436 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:40.924902 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e5e32f59-16e1-486a-b2ef-9882f3d08aa3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059593 Jul 11 14:45:40.925880 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 372/764] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:45:40 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:45:41.089837 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-4fbdf5ce-be8c-45c3-b7f4-df4e63923436 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:41.104705 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-4fbdf5ce-be8c-45c3-b7f4-df4e63923436 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1685 microversion: 2.1 time: 0.202152 Jul 11 14:45:41.105541 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 393/765] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:40 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1685 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:45:41.212978 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-c3539e27-6dca-40df-a5dd-0bc8224acd85 req-64f90c3e-4820-48db-9423-ae7bd49179ef service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cb814dfe-6200-40ed-832f-df4face7a16e", "tag": "acddbdf4-6edc-4eaf-8287-a34936c1b912"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:41.221716 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-c3539e27-6dca-40df-a5dd-0bc8224acd85 req-64f90c3e-4820-48db-9423-ae7bd49179ef service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:41.222157 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-c3539e27-6dca-40df-a5dd-0bc8224acd85 req-64f90c3e-4820-48db-9423-ae7bd49179ef service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:41.235034 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-c3539e27-6dca-40df-a5dd-0bc8224acd85 req-64f90c3e-4820-48db-9423-ae7bd49179ef service nova] Creating event network-changed:acddbdf4-6edc-4eaf-8287-a34936c1b912 for instance cb814dfe-6200-40ed-832f-df4face7a16e on hv-cinder-maste Jul 11 14:45:41.240260 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-c3539e27-6dca-40df-a5dd-0bc8224acd85 req-64f90c3e-4820-48db-9423-ae7bd49179ef service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.102406 Jul 11 14:45:41.240884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 373/766] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:45:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:45:41.310755 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1abf8bc0-a313-46b4-8ef6-9aa80a41128d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:41.311374 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1abf8bc0-a313-46b4-8ef6-9aa80a41128d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:41.318563 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1abf8bc0-a313-46b4-8ef6-9aa80a41128d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:41.319021 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1abf8bc0-a313-46b4-8ef6-9aa80a41128d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:41.319791 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1abf8bc0-a313-46b4-8ef6-9aa80a41128d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:41.320227 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1abf8bc0-a313-46b4-8ef6-9aa80a41128d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:41.470864 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1abf8bc0-a313-46b4-8ef6-9aa80a41128d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:41.482780 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1abf8bc0-a313-46b4-8ef6-9aa80a41128d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1267 microversion: 2.1 time: 0.176531 Jul 11 14:45:41.483592 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 394/767] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:41 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1267 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:41.938259 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7cee1730-f99c-4efc-92ce-ccd0b0d4e8ae tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:41.939253 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-7cee1730-f99c-4efc-92ce-ccd0b0d4e8ae tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:41.946482 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7cee1730-f99c-4efc-92ce-ccd0b0d4e8ae tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:41.947024 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7cee1730-f99c-4efc-92ce-ccd0b0d4e8ae tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:41.947900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7cee1730-f99c-4efc-92ce-ccd0b0d4e8ae tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:41.948340 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7cee1730-f99c-4efc-92ce-ccd0b0d4e8ae tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:41.988242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-7cee1730-f99c-4efc-92ce-ccd0b0d4e8ae tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054878 Jul 11 14:45:41.988960 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 374/768] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:45:41 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:45:42.126099 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f222e30b-0ae8-4aa9-96b1-6ca32bef3178 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:42.128088 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f222e30b-0ae8-4aa9-96b1-6ca32bef3178 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:42.139901 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f222e30b-0ae8-4aa9-96b1-6ca32bef3178 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:42.141000 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f222e30b-0ae8-4aa9-96b1-6ca32bef3178 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:42.141447 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f222e30b-0ae8-4aa9-96b1-6ca32bef3178 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:42.141894 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f222e30b-0ae8-4aa9-96b1-6ca32bef3178 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:42.309661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f222e30b-0ae8-4aa9-96b1-6ca32bef3178 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:42.319976 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f222e30b-0ae8-4aa9-96b1-6ca32bef3178 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1685 microversion: 2.1 time: 0.201907 Jul 11 14:45:42.320702 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 395/769] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:42 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1685 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:42.500575 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a8de696a-b10d-41b9-839c-d2aa2380bf96 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:42.501540 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a8de696a-b10d-41b9-839c-d2aa2380bf96 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:42.509489 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a8de696a-b10d-41b9-839c-d2aa2380bf96 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:42.509942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a8de696a-b10d-41b9-839c-d2aa2380bf96 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:42.510798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a8de696a-b10d-41b9-839c-d2aa2380bf96 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:42.511239 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a8de696a-b10d-41b9-839c-d2aa2380bf96 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:42.663848 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a8de696a-b10d-41b9-839c-d2aa2380bf96 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:42.676037 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a8de696a-b10d-41b9-839c-d2aa2380bf96 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1267 microversion: 2.1 time: 0.179215 Jul 11 14:45:42.677274 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 375/770] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:42 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1267 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:45:43.001370 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-bc548afa-de56-4dce-93f2-468a5a2c1f94 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:43.002409 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-bc548afa-de56-4dce-93f2-468a5a2c1f94 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:43.009735 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bc548afa-de56-4dce-93f2-468a5a2c1f94 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:43.010239 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bc548afa-de56-4dce-93f2-468a5a2c1f94 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:43.011186 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bc548afa-de56-4dce-93f2-468a5a2c1f94 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:43.011672 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bc548afa-de56-4dce-93f2-468a5a2c1f94 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:43.044922 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-bc548afa-de56-4dce-93f2-468a5a2c1f94 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047618 Jul 11 14:45:43.045423 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 396/771] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:45:42 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:45:43.347728 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c101afc9-0f1c-4b45-a4f7-b53fb7328c20 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:43.348698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c101afc9-0f1c-4b45-a4f7-b53fb7328c20 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:43.356519 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c101afc9-0f1c-4b45-a4f7-b53fb7328c20 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:43.356988 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c101afc9-0f1c-4b45-a4f7-b53fb7328c20 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:43.357806 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c101afc9-0f1c-4b45-a4f7-b53fb7328c20 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:43.358255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c101afc9-0f1c-4b45-a4f7-b53fb7328c20 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:43.541894 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c101afc9-0f1c-4b45-a4f7-b53fb7328c20 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:43.555572 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c101afc9-0f1c-4b45-a4f7-b53fb7328c20 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1685 microversion: 2.1 time: 0.213109 Jul 11 14:45:43.556657 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 376/772] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:43 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1685 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:45:43.694285 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f4f0c29b-f00f-4a5d-87bb-ec6c861004f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:43.695684 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f4f0c29b-f00f-4a5d-87bb-ec6c861004f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:43.705378 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f4f0c29b-f00f-4a5d-87bb-ec6c861004f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:43.706122 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f4f0c29b-f00f-4a5d-87bb-ec6c861004f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:43.708105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f4f0c29b-f00f-4a5d-87bb-ec6c861004f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:43.709193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f4f0c29b-f00f-4a5d-87bb-ec6c861004f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:43.949388 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f4f0c29b-f00f-4a5d-87bb-ec6c861004f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:43.958616 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f4f0c29b-f00f-4a5d-87bb-ec6c861004f6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1267 microversion: 2.1 time: 0.268385 Jul 11 14:45:43.959229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 397/773] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:43 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1267 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:44.060634 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-44032892-576e-4c86-b5b5-02802aea1be1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:44.062184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-44032892-576e-4c86-b5b5-02802aea1be1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:44.073279 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-44032892-576e-4c86-b5b5-02802aea1be1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:44.074039 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-44032892-576e-4c86-b5b5-02802aea1be1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:44.074764 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-44032892-576e-4c86-b5b5-02802aea1be1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:44.075345 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-44032892-576e-4c86-b5b5-02802aea1be1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:44.113763 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-44032892-576e-4c86-b5b5-02802aea1be1 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.058114 Jul 11 14:45:44.114824 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 377/774] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:45:44 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21/os-volume_attachments => generated 25 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 11 14:45:44.219998 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-4f307edf-488c-42b6-a395-bf8b2b3b89fb tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:44.220513 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-4f307edf-488c-42b6-a395-bf8b2b3b89fb tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:44.228228 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4f307edf-488c-42b6-a395-bf8b2b3b89fb tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:44.228855 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4f307edf-488c-42b6-a395-bf8b2b3b89fb tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:44.229527 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4f307edf-488c-42b6-a395-bf8b2b3b89fb tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:44.230174 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4f307edf-488c-42b6-a395-bf8b2b3b89fb tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:44.268928 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-4f307edf-488c-42b6-a395-bf8b2b3b89fb tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Going to try to terminate instance {{(pid=130159) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:45:44.382229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-4f307edf-488c-42b6-a395-bf8b2b3b89fb tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "DELETE /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 204 len: 0 microversion: 2.1 time: 0.165896 Jul 11 14:45:44.383392 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 398/775] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:45:44 2022] DELETE /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 11 14:45:44.392991 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-adda7db0-0c94-402f-9fa3-826979d3d6e3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:44.394023 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-adda7db0-0c94-402f-9fa3-826979d3d6e3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:44.402744 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-adda7db0-0c94-402f-9fa3-826979d3d6e3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:44.403420 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-adda7db0-0c94-402f-9fa3-826979d3d6e3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:44.404345 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-adda7db0-0c94-402f-9fa3-826979d3d6e3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:44.404917 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-adda7db0-0c94-402f-9fa3-826979d3d6e3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:44.579385 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-ea88bab6-0fec-40f7-904c-d07763b99420 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:44.580714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-ea88bab6-0fec-40f7-904c-d07763b99420 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:44.590943 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ea88bab6-0fec-40f7-904c-d07763b99420 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:44.591546 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ea88bab6-0fec-40f7-904c-d07763b99420 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:44.592687 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ea88bab6-0fec-40f7-904c-d07763b99420 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:44.593241 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ea88bab6-0fec-40f7-904c-d07763b99420 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:44.665168 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-adda7db0-0c94-402f-9fa3-826979d3d6e3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:44.686480 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-ea88bab6-0fec-40f7-904c-d07763b99420 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:44.696638 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-adda7db0-0c94-402f-9fa3-826979d3d6e3 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 200 len: 1682 microversion: 2.1 time: 0.308502 Jul 11 14:45:44.707125 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-ea88bab6-0fec-40f7-904c-d07763b99420 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1618 microversion: 2.1 time: 0.135008 Jul 11 14:45:44.710249 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 378/777] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:44 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 1682 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:44.710486 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 399/777] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:44 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1618 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:44.911618 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-66f54f64-ab3e-418f-af8d-a4d3f2a1c43d req-e0fb8571-66ba-46bd-95e4-8b886c990a6c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f66823cd-4fe8-42cf-ab7a-d7cd7afc289a", "name": "network-vif-deleted", "tag": "5a30e035-3f54-418c-8ac6-380a40fc2f5d"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:44.925877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-66f54f64-ab3e-418f-af8d-a4d3f2a1c43d req-e0fb8571-66ba-46bd-95e4-8b886c990a6c service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:44.926330 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-66f54f64-ab3e-418f-af8d-a4d3f2a1c43d req-e0fb8571-66ba-46bd-95e4-8b886c990a6c service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:44.940033 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-66f54f64-ab3e-418f-af8d-a4d3f2a1c43d req-e0fb8571-66ba-46bd-95e4-8b886c990a6c service nova] Creating event network-vif-deleted:5a30e035-3f54-418c-8ac6-380a40fc2f5d for instance f66823cd-4fe8-42cf-ab7a-d7cd7afc289a on hv-cinder-maste Jul 11 14:45:44.946855 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-66f54f64-ab3e-418f-af8d-a4d3f2a1c43d req-e0fb8571-66ba-46bd-95e4-8b886c990a6c service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040945 Jul 11 14:45:44.947894 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 400/778] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:45:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:45:44.977635 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-34e69a3a-f50f-47f5-b5c8-b47e41595c2f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:44.978954 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-34e69a3a-f50f-47f5-b5c8-b47e41595c2f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:44.988968 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-34e69a3a-f50f-47f5-b5c8-b47e41595c2f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:44.990222 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-34e69a3a-f50f-47f5-b5c8-b47e41595c2f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:44.991272 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-34e69a3a-f50f-47f5-b5c8-b47e41595c2f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:44.991965 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-34e69a3a-f50f-47f5-b5c8-b47e41595c2f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:45.180401 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-34e69a3a-f50f-47f5-b5c8-b47e41595c2f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:45.188819 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-34e69a3a-f50f-47f5-b5c8-b47e41595c2f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1267 microversion: 2.1 time: 0.216474 Jul 11 14:45:45.189628 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 379/779] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:44 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1267 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:45.718933 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-446ce929-8408-4a1f-9e97-e61609c56a07 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:45.719826 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-446ce929-8408-4a1f-9e97-e61609c56a07 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:45.726939 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-446ce929-8408-4a1f-9e97-e61609c56a07 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:45.727388 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-446ce929-8408-4a1f-9e97-e61609c56a07 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:45.728188 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-446ce929-8408-4a1f-9e97-e61609c56a07 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:45.729094 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-446ce929-8408-4a1f-9e97-e61609c56a07 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:45.740407 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-cbe1def4-5a8e-42bd-986e-af36803c68c2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:45.741588 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-cbe1def4-5a8e-42bd-986e-af36803c68c2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:45.753332 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cbe1def4-5a8e-42bd-986e-af36803c68c2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:45.754131 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cbe1def4-5a8e-42bd-986e-af36803c68c2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:45.756213 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cbe1def4-5a8e-42bd-986e-af36803c68c2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:45.756977 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cbe1def4-5a8e-42bd-986e-af36803c68c2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:45.841019 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-cbe1def4-5a8e-42bd-986e-af36803c68c2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:45.850851 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-cbe1def4-5a8e-42bd-986e-af36803c68c2 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1618 microversion: 2.1 time: 0.119506 Jul 11 14:45:45.851781 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 380/780] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:45 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1618 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:45.947801 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-446ce929-8408-4a1f-9e97-e61609c56a07 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:45.957191 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-446ce929-8408-4a1f-9e97-e61609c56a07 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 200 len: 1682 microversion: 2.1 time: 0.243037 Jul 11 14:45:45.957824 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 401/781] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:45 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 1682 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:46.214474 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-52bcf3d9-8f15-47cf-9b4f-1c9bfbe10e1a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:46.215095 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-52bcf3d9-8f15-47cf-9b4f-1c9bfbe10e1a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:46.222499 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-52bcf3d9-8f15-47cf-9b4f-1c9bfbe10e1a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:46.222961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-52bcf3d9-8f15-47cf-9b4f-1c9bfbe10e1a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:46.224309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-52bcf3d9-8f15-47cf-9b4f-1c9bfbe10e1a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:46.224755 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-52bcf3d9-8f15-47cf-9b4f-1c9bfbe10e1a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:46.405773 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-52bcf3d9-8f15-47cf-9b4f-1c9bfbe10e1a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:46.415324 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-52bcf3d9-8f15-47cf-9b4f-1c9bfbe10e1a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1267 microversion: 2.1 time: 0.205002 Jul 11 14:45:46.416105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 381/782] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:46 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1267 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:46.876437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3b7ce345-4dcf-427c-ba34-8edec1df6c04 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:46.877062 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3b7ce345-4dcf-427c-ba34-8edec1df6c04 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:46.885325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3b7ce345-4dcf-427c-ba34-8edec1df6c04 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:46.885778 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3b7ce345-4dcf-427c-ba34-8edec1df6c04 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:46.886655 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3b7ce345-4dcf-427c-ba34-8edec1df6c04 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:46.887313 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3b7ce345-4dcf-427c-ba34-8edec1df6c04 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:46.957034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3b7ce345-4dcf-427c-ba34-8edec1df6c04 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:46.966645 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3b7ce345-4dcf-427c-ba34-8edec1df6c04 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 200 len: 1327 microversion: 2.1 time: 0.094680 Jul 11 14:45:46.967409 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 402/783] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:46 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 1327 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:46.975078 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-97cfa194-1b5b-48a4-886e-0731360fb072 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:46.975980 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-97cfa194-1b5b-48a4-886e-0731360fb072 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:46.982541 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-97cfa194-1b5b-48a4-886e-0731360fb072 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:46.982981 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-97cfa194-1b5b-48a4-886e-0731360fb072 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:46.983761 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-97cfa194-1b5b-48a4-886e-0731360fb072 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:46.984192 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-97cfa194-1b5b-48a4-886e-0731360fb072 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:47.407472 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-97cfa194-1b5b-48a4-886e-0731360fb072 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:47.416559 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-97cfa194-1b5b-48a4-886e-0731360fb072 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 200 len: 1682 microversion: 2.1 time: 0.445569 Jul 11 14:45:47.417161 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 382/784] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:46 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 1682 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:47.434939 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3b5c51be-9f29-4fa1-a14d-eddf6dc947ae tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:47.435548 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3b5c51be-9f29-4fa1-a14d-eddf6dc947ae tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:47.442227 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3b5c51be-9f29-4fa1-a14d-eddf6dc947ae tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:47.442739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3b5c51be-9f29-4fa1-a14d-eddf6dc947ae tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:47.443735 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3b5c51be-9f29-4fa1-a14d-eddf6dc947ae tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:47.444274 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3b5c51be-9f29-4fa1-a14d-eddf6dc947ae tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:47.467726 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:47.604632 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3b5c51be-9f29-4fa1-a14d-eddf6dc947ae tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:47.613052 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3b5c51be-9f29-4fa1-a14d-eddf6dc947ae tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1453 microversion: 2.1 time: 0.183329 Jul 11 14:45:47.613947 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 403/785] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:47 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1453 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:47.638869 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-eab96316-2d75-450c-86d5-db34d174d313 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:47.639889 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-eab96316-2d75-450c-86d5-db34d174d313 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:47.648059 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eab96316-2d75-450c-86d5-db34d174d313 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:47.648710 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eab96316-2d75-450c-86d5-db34d174d313 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:47.649980 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eab96316-2d75-450c-86d5-db34d174d313 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:47.650571 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eab96316-2d75-450c-86d5-db34d174d313 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:47.964808 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-eab96316-2d75-450c-86d5-db34d174d313 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:47.972965 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-eab96316-2d75-450c-86d5-db34d174d313 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1453 microversion: 2.1 time: 0.339738 Jul 11 14:45:47.974113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 383/786] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:47 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1453 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:47.995375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c091afc9-870f-45ed-a090-451f7182fbbe tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:47.996862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c091afc9-870f-45ed-a090-451f7182fbbe tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: f66823cd-4fe8-42cf-ab7a-d7cd7afc289a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:48.007772 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c091afc9-870f-45ed-a090-451f7182fbbe tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:48.008761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c091afc9-870f-45ed-a090-451f7182fbbe tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:48.010415 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c091afc9-870f-45ed-a090-451f7182fbbe tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:48.011356 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c091afc9-870f-45ed-a090-451f7182fbbe tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:48.032443 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-c091afc9-870f-45ed-a090-451f7182fbbe tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] HTTP exception thrown: Instance f66823cd-4fe8-42cf-ab7a-d7cd7afc289a could not be found. Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c091afc9-870f-45ed-a090-451f7182fbbe tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Request method failure captured: Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a HTTP/1.1 Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZVO0vcEhdpPwwfJZnu1y3p7Tg0Q8GENrUVh0NohR9oAPNIjKFLY8v62KxMOiNHGfwF-EVGge1hPwlyTCX1uExRac1NToAISnaSDmVmpVWA-TKGS1rzgSZVkfiPxYTNCyZmEJBrMLK9NH_pMk0kTrm35o7KBc7uMvJqd_pTjsh0TKE Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: 98966d44ea88477b94215761cdc92ea7 Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1829929608 Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:45:48.035278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:45:48.040237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/98966d44ea88477b94215761cdc92ea7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/98966d44ea88477b94215761cdc92ea7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_98966d44ea88477b94215761cdc92ea7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/98966d44ea88477b94215761cdc92ea7"}]}] Jul 11 14:45:48.040237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1829929608 Jul 11 14:45:48.040237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: 98966d44ea88477b94215761cdc92ea7 Jul 11 14:45:48.040237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1829929608 Jul 11 14:45:48.040237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-ServerRescueNegativeTestJSON-1829929608-project Jul 11 14:45:48.040237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:45:48.040237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:45:48.040237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: fa478fb4b41e49c5b1f9291f925cca23 Jul 11 14:45:48.040237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1829929608-project Jul 11 14:45:48.040237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:45:48.040237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance f66823cd-4fe8-42cf-ab7a-d7cd7afc289a could not be found. Jul 11 14:45:48.040237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': 'f66823cd-4fe8-42cf-ab7a-d7cd7afc289a'} Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f66823cd-4fe8-42cf-ab7a-d7cd7afc289a could not be found. Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:45:48.041813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:45:48.044389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:45:48.044389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:45:48.044389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:45:48.044389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:45:48.044389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:45:48.044389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:45:48.044389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:45:48.044389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f66823cd-4fe8-42cf-ab7a-d7cd7afc289a could not be found. Jul 11 14:45:48.044389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:48.044389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:45:48.044389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:48.044389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:48.044389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:45:48.044389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:45:48.044389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:45:48.044389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:45:48.044389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f66823cd-4fe8-42cf-ab7a-d7cd7afc289a could not be found. Jul 11 14:45:48.044389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:48.046776 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c091afc9-870f-45ed-a090-451f7182fbbe tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Returning 404 to user: Instance f66823cd-4fe8-42cf-ab7a-d7cd7afc289a could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:45:48.046776 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c091afc9-870f-45ed-a090-451f7182fbbe tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a" status: 404 len: 111 microversion: 2.1 time: 0.050236 Jul 11 14:45:48.046776 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 404/787] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:47 2022] GET /compute/v2.1/servers/f66823cd-4fe8-42cf-ab7a-d7cd7afc289a => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:45:48.055140 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-23d19c14-e69c-428c-8380-8ee646eac038 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '.version_select at 0x7f59300618b0>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:48.079236 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-23d19c14-e69c-428c-8380-8ee646eac038 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-126367978" status: 202 len: 0 microversion: 2.1 time: 0.030023 Jul 11 14:45:48.080086 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 384/788] 10.222.0.72 () {58 vars in 1217 bytes} [Mon Jul 11 14:45:48 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-126367978 => generated 0 bytes in 32 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:45:48.434732 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3947f1eb-7a0c-4de0-ac3d-b073c21dba31 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:48.435450 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3947f1eb-7a0c-4de0-ac3d-b073c21dba31 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:48.443575 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3947f1eb-7a0c-4de0-ac3d-b073c21dba31 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:48.444369 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3947f1eb-7a0c-4de0-ac3d-b073c21dba31 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:48.445172 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3947f1eb-7a0c-4de0-ac3d-b073c21dba31 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:48.445790 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3947f1eb-7a0c-4de0-ac3d-b073c21dba31 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:48.644501 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3947f1eb-7a0c-4de0-ac3d-b073c21dba31 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:48.711390 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3947f1eb-7a0c-4de0-ac3d-b073c21dba31 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 200 len: 1614 microversion: 2.1 time: 0.266023 Jul 11 14:45:48.713376 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 405/789] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:48 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 1614 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:48.769948 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-d5254ca8-36d3-4b02-9b71-5d10cc404b08 req-0698de4e-81d8-468f-a310-00f65784f38a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1f161996-6f14-46a9-9200-c8763cb67e21", "name": "network-vif-deleted", "tag": "2ea3ae8a-9690-4d6d-82ab-c4549ae55695"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:48.787196 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-d5254ca8-36d3-4b02-9b71-5d10cc404b08 req-0698de4e-81d8-468f-a310-00f65784f38a service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:48.789577 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-d5254ca8-36d3-4b02-9b71-5d10cc404b08 req-0698de4e-81d8-468f-a310-00f65784f38a service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:48.830372 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-d5254ca8-36d3-4b02-9b71-5d10cc404b08 req-0698de4e-81d8-468f-a310-00f65784f38a service nova] Creating event network-vif-deleted:2ea3ae8a-9690-4d6d-82ab-c4549ae55695 for instance 1f161996-6f14-46a9-9200-c8763cb67e21 on hv-cinder-maste Jul 11 14:45:48.844814 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-d5254ca8-36d3-4b02-9b71-5d10cc404b08 req-0698de4e-81d8-468f-a310-00f65784f38a service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.081161 Jul 11 14:45:48.845442 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 385/790] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:45:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:45:49.740139 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-68902e0f-7304-496a-8af1-71f445c38a6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:49.741051 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-68902e0f-7304-496a-8af1-71f445c38a6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:49.749064 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-68902e0f-7304-496a-8af1-71f445c38a6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:49.749774 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-68902e0f-7304-496a-8af1-71f445c38a6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:49.750692 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-68902e0f-7304-496a-8af1-71f445c38a6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:49.751164 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-68902e0f-7304-496a-8af1-71f445c38a6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:49.776832 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-68902e0f-7304-496a-8af1-71f445c38a6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] HTTP exception thrown: Instance 1f161996-6f14-46a9-9200-c8763cb67e21 could not be found. Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-68902e0f-7304-496a-8af1-71f445c38a6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Request method failure captured: Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 HTTP/1.1 Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZbQUJdl7M18phk4FM9-GGzi8XbGx24YhEK9KsOfI165n5v27J9wq7wzJjs1m7qkLCobj_5JRaSFQlNcVRL5jfwSqdoh1nodKIJxEOpM9gSJP97YpvVDnXLPsMNjwZQwG04QTf0uG5zLXZ7eKV_ivBXCJkCsdmhsPnE9ln-9M7wHs4 Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:45:49.778483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:45:49.781023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}] Jul 11 14:45:49.781023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:45:49.781023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:45:49.781023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:45:49.781023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:45:49.781023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:45:49.781023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:45:49.781023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 3ae56fd1172f4bf8806eefc6e8877fef Jul 11 14:45:49.781023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:45:49.781023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:45:49.781023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance 1f161996-6f14-46a9-9200-c8763cb67e21 could not be found. Jul 11 14:45:49.781023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '1f161996-6f14-46a9-9200-c8763cb67e21'} Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1f161996-6f14-46a9-9200-c8763cb67e21 could not be found. Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:45:49.782800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:45:49.786045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:45:49.786045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:45:49.786045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:45:49.786045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:45:49.786045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:45:49.786045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:45:49.786045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:45:49.786045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1f161996-6f14-46a9-9200-c8763cb67e21 could not be found. Jul 11 14:45:49.786045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:49.786045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:45:49.786045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:49.786045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:49.786045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:45:49.786045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:45:49.786045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:45:49.786045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:45:49.786045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1f161996-6f14-46a9-9200-c8763cb67e21 could not be found. Jul 11 14:45:49.786045 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:45:49.792508 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-68902e0f-7304-496a-8af1-71f445c38a6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Returning 404 to user: Instance 1f161996-6f14-46a9-9200-c8763cb67e21 could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:45:49.792508 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-68902e0f-7304-496a-8af1-71f445c38a6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 404 len: 111 microversion: 2.1 time: 0.044205 Jul 11 14:45:49.792508 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 406/791] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:49 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 111 bytes in 54 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 11 14:45:49.793276 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3948c9f7-d29a-470b-9818-d86576f41d85 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '.version_select at 0x7f5930061940>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:49.824821 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-3948c9f7-d29a-470b-9818-d86576f41d85 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2057892801" status: 202 len: 0 microversion: 2.1 time: 0.038084 Jul 11 14:45:49.825892 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 386/792] 10.222.0.72 () {58 vars in 1220 bytes} [Mon Jul 11 14:45:49 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2057892801 => generated 0 bytes in 41 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:45:50.862322 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8d70f845-4272-4253-bd88-84351d93d643 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Action: 'create', calling method: .version_select at 0x7f592af7bca0>, body: {"keypair": {"name": "tempest-keypair-1469998488", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNYUQqBkQSr5KRs0OroWDObJFhQRTb0aRUZ6OVDpbfWiqQbBMpZrvYMJeSvgTsjAUgze/uAMiiGkCyKlgRmxdVzdYuzftm/PHW7qkF9fxIDp7xNDQRXVn+fB5FMNR3CNdg=="}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:50.881408 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-8d70f845-4272-4253-bd88-84351d93d643 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Getting quotas for project 1baa6ea2e19747d1af1e1042c362895d. Resources: {'key_pairs'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:45:50.886726 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-8d70f845-4272-4253-bd88-84351d93d643 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Getting quotas for user 3ae56fd1172f4bf8806eefc6e8877fef and project 1baa6ea2e19747d1af1e1042c362895d. Resources: {'key_pairs'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:45:50.916061 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8d70f845-4272-4253-bd88-84351d93d643 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.058937 Jul 11 14:45:50.917297 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 407/793] 10.222.0.72 () {60 vars in 1158 bytes} [Mon Jul 11 14:45:50 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:45:51.271685 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-fab941c2-ca6e-4989-89b3-1f28f4f91645 req-f7f5ee19-18d7-456e-9ea1-6150e1f79485 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1009499c-a8c8-472c-bef5-fdf42c002c0b", "tag": "1544341d-a8ad-41b0-8bc7-e3d71545d58f"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:51.284355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-fab941c2-ca6e-4989-89b3-1f28f4f91645 req-f7f5ee19-18d7-456e-9ea1-6150e1f79485 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:51.285379 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-fab941c2-ca6e-4989-89b3-1f28f4f91645 req-f7f5ee19-18d7-456e-9ea1-6150e1f79485 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:51.300081 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-fab941c2-ca6e-4989-89b3-1f28f4f91645 req-f7f5ee19-18d7-456e-9ea1-6150e1f79485 service nova] Creating event network-changed:1544341d-a8ad-41b0-8bc7-e3d71545d58f for instance 1009499c-a8c8-472c-bef5-fdf42c002c0b on hv-cinder-maste Jul 11 14:45:51.307396 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-fab941c2-ca6e-4989-89b3-1f28f4f91645 req-f7f5ee19-18d7-456e-9ea1-6150e1f79485 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.149885 Jul 11 14:45:51.308177 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 387/794] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:45:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:45:52.574056 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-7b8d2947-117e-49f7-a12f-f4578c53362a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:52.574945 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-7b8d2947-117e-49f7-a12f-f4578c53362a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:52.583579 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7b8d2947-117e-49f7-a12f-f4578c53362a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:52.584475 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7b8d2947-117e-49f7-a12f-f4578c53362a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:52.585578 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7b8d2947-117e-49f7-a12f-f4578c53362a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:52.586218 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7b8d2947-117e-49f7-a12f-f4578c53362a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:52.636526 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-7b8d2947-117e-49f7-a12f-f4578c53362a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Going to try to terminate instance {{(pid=130159) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:45:52.792604 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-7b8d2947-117e-49f7-a12f-f4578c53362a tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "DELETE /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 204 len: 0 microversion: 2.1 time: 0.222460 Jul 11 14:45:52.799649 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 408/795] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:45:52 2022] DELETE /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 11 14:45:52.809884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-40b219c1-20a0-4ec2-b477-6c7ce5ddcbaf tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:52.810844 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-40b219c1-20a0-4ec2-b477-6c7ce5ddcbaf tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:52.818064 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-40b219c1-20a0-4ec2-b477-6c7ce5ddcbaf tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:52.818552 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-40b219c1-20a0-4ec2-b477-6c7ce5ddcbaf tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:52.819340 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-40b219c1-20a0-4ec2-b477-6c7ce5ddcbaf tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:52.819781 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-40b219c1-20a0-4ec2-b477-6c7ce5ddcbaf tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:53.025349 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-40b219c1-20a0-4ec2-b477-6c7ce5ddcbaf tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:53.033971 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-40b219c1-20a0-4ec2-b477-6c7ce5ddcbaf tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1507 microversion: 2.1 time: 0.231867 Jul 11 14:45:53.034821 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 388/796] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:52 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1507 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:45:53.036319 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2484aea8-e941-4b2b-947e-835638b3c37b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1145856964", "imageRef": "a588fa24-23b5-4e53-87c6-7e600ee3a5ef", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8679b039-4e4e-4ae1-83e8-21c78c675a96"}], "security_groups": [{"name": "tempest-securitygroup--1442434882"}], "key_name": "tempest-keypair-1469998488", "user_data": "}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:53.186561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.network.neutron [None req-2484aea8-e941-4b2b-947e-835638b3c37b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] validate_networks() for [('8679b039-4e4e-4ae1-83e8-21c78c675a96', None, None, None, None, None)] {{(pid=130159) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:45:53.667423 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-2484aea8-e941-4b2b-947e-835638b3c37b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Flavor limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:45:53.668519 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-2484aea8-e941-4b2b-947e-835638b3c37b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Image limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:45:53.668909 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-2484aea8-e941-4b2b-947e-835638b3c37b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Flavor pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:45:53.669473 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-2484aea8-e941-4b2b-947e-835638b3c37b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Image pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:45:53.669998 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-2484aea8-e941-4b2b-947e-835638b3c37b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:45:53.681082 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2484aea8-e941-4b2b-947e-835638b3c37b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:53.681529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2484aea8-e941-4b2b-947e-835638b3c37b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:53.682357 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2484aea8-e941-4b2b-947e-835638b3c37b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:53.682833 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2484aea8-e941-4b2b-947e-835638b3c37b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:53.702814 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-2484aea8-e941-4b2b-947e-835638b3c37b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Getting quotas for project 1baa6ea2e19747d1af1e1042c362895d. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:45:53.708597 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-2484aea8-e941-4b2b-947e-835638b3c37b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Getting quotas for user 3ae56fd1172f4bf8806eefc6e8877fef and project 1baa6ea2e19747d1af1e1042c362895d. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:45:53.721817 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2484aea8-e941-4b2b-947e-835638b3c37b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Going to run 1 instances... {{(pid=130159) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:45:53.724534 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2484aea8-e941-4b2b-947e-835638b3c37b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a588fa24-23b5-4e53-87c6-7e600ee3a5ef',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130159) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:45:53.775601 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2484aea8-e941-4b2b-947e-835638b3c37b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.744852 Jul 11 14:45:53.776282 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 409/797] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:45:53 2022] POST /compute/v2.1/servers => generated 402 bytes in 747 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 11 14:45:53.790990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2a3ddf07-8871-4db7-94ec-c8cdcf812291 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:53.792060 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2a3ddf07-8871-4db7-94ec-c8cdcf812291 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:53.870776 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2a3ddf07-8871-4db7-94ec-c8cdcf812291 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:53.878824 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2a3ddf07-8871-4db7-94ec-c8cdcf812291 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 200 len: 1242 microversion: 2.1 time: 0.095268 Jul 11 14:45:53.879464 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 389/798] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:53 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 1242 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:54.059495 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-56cf6d85-75e0-4f60-9d91-6730842677d0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:54.060130 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-56cf6d85-75e0-4f60-9d91-6730842677d0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:54.063628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=94,nova_api:UPDATE=2,nova_api:INSERT=4 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:54.069216 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-56cf6d85-75e0-4f60-9d91-6730842677d0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:54.069680 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-56cf6d85-75e0-4f60-9d91-6730842677d0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:54.070550 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-56cf6d85-75e0-4f60-9d91-6730842677d0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:54.071022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-56cf6d85-75e0-4f60-9d91-6730842677d0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:54.249484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-56cf6d85-75e0-4f60-9d91-6730842677d0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:54.258243 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-56cf6d85-75e0-4f60-9d91-6730842677d0 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1507 microversion: 2.1 time: 0.202767 Jul 11 14:45:54.258908 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 410/799] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:54 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1507 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:45:54.905525 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d9cda793-2f24-486c-95d1-e1069c10b119 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:54.906589 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d9cda793-2f24-486c-95d1-e1069c10b119 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:54.915540 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d9cda793-2f24-486c-95d1-e1069c10b119 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:54.916142 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d9cda793-2f24-486c-95d1-e1069c10b119 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:54.917082 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d9cda793-2f24-486c-95d1-e1069c10b119 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:54.917644 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d9cda793-2f24-486c-95d1-e1069c10b119 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:55.012374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d9cda793-2f24-486c-95d1-e1069c10b119 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:55.027942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d9cda793-2f24-486c-95d1-e1069c10b119 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 200 len: 1294 microversion: 2.1 time: 0.126242 Jul 11 14:45:55.028894 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 390/800] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:54 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 1294 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:45:55.279704 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9b4c65de-ecb2-4e72-ae4e-5554d29ee48e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:55.281670 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9b4c65de-ecb2-4e72-ae4e-5554d29ee48e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:55.294685 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9b4c65de-ecb2-4e72-ae4e-5554d29ee48e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:55.298777 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9b4c65de-ecb2-4e72-ae4e-5554d29ee48e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:55.300960 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9b4c65de-ecb2-4e72-ae4e-5554d29ee48e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:55.303853 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9b4c65de-ecb2-4e72-ae4e-5554d29ee48e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:55.420700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-1a942726-29db-42b0-b387-eac9934939bb req-19840102-2489-4e6c-824d-ffa50a392a6b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cc0591e3-09b9-41cd-b222-f730282f8572", "name": "network-vif-deleted", "tag": "9af04c14-ce8d-46f1-8097-0f76fe117ff9"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:55.454610 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-1a942726-29db-42b0-b387-eac9934939bb req-19840102-2489-4e6c-824d-ffa50a392a6b service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:55.455704 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-1a942726-29db-42b0-b387-eac9934939bb req-19840102-2489-4e6c-824d-ffa50a392a6b service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:55.471008 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9b4c65de-ecb2-4e72-ae4e-5554d29ee48e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:55.490089 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9b4c65de-ecb2-4e72-ae4e-5554d29ee48e tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 200 len: 1465 microversion: 2.1 time: 0.214027 Jul 11 14:45:55.505842 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 411/801] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:55 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 1465 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:55.506388 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-1a942726-29db-42b0-b387-eac9934939bb req-19840102-2489-4e6c-824d-ffa50a392a6b service nova] Creating event network-vif-deleted:9af04c14-ce8d-46f1-8097-0f76fe117ff9 for instance cc0591e3-09b9-41cd-b222-f730282f8572 on hv-cinder-maste Jul 11 14:45:55.528946 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-1a942726-29db-42b0-b387-eac9934939bb req-19840102-2489-4e6c-824d-ffa50a392a6b service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.122004 Jul 11 14:45:55.529996 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 391/802] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:45:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:45:56.049779 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a93425fd-ba6b-4a54-b6b6-449329278e71 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:56.051108 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a93425fd-ba6b-4a54-b6b6-449329278e71 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:56.060659 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a93425fd-ba6b-4a54-b6b6-449329278e71 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:56.061219 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a93425fd-ba6b-4a54-b6b6-449329278e71 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:56.062121 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a93425fd-ba6b-4a54-b6b6-449329278e71 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:56.062776 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a93425fd-ba6b-4a54-b6b6-449329278e71 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:56.489552 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a93425fd-ba6b-4a54-b6b6-449329278e71 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:56.500639 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a93425fd-ba6b-4a54-b6b6-449329278e71 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 200 len: 1368 microversion: 2.1 time: 0.457230 Jul 11 14:45:56.501550 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 412/803] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:56 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 1368 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:56.532401 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a2ff2b48-862c-41b6-8cc0-3cdfdc7d5e94 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:56.534882 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a2ff2b48-862c-41b6-8cc0-3cdfdc7d5e94 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: cc0591e3-09b9-41cd-b222-f730282f8572] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:56.550755 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a2ff2b48-862c-41b6-8cc0-3cdfdc7d5e94 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:56.551862 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a2ff2b48-862c-41b6-8cc0-3cdfdc7d5e94 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:56.553455 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a2ff2b48-862c-41b6-8cc0-3cdfdc7d5e94 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:56.554600 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a2ff2b48-862c-41b6-8cc0-3cdfdc7d5e94 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:56.595183 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-a2ff2b48-862c-41b6-8cc0-3cdfdc7d5e94 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] HTTP exception thrown: Instance cc0591e3-09b9-41cd-b222-f730282f8572 could not be found. Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a2ff2b48-862c-41b6-8cc0-3cdfdc7d5e94 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Request method failure captured: Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 HTTP/1.1 Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDZVO0vcEhdpPwwfJZnu1y3p7Tg0Q8GENrUVh0NohR9oAPNIjKFLY8v62KxMOiNHGfwF-EVGge1hPwlyTCX1uExRac1NToAISnaSDmVmpVWA-TKGS1rzgSZVkfiPxYTNCyZmEJBrMLK9NH_pMk0kTrm35o7KBc7uMvJqd_pTjsh0TKE Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: 98966d44ea88477b94215761cdc92ea7 Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1829929608 Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:45:56.596920 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:45:56.599587 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/98966d44ea88477b94215761cdc92ea7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/98966d44ea88477b94215761cdc92ea7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_98966d44ea88477b94215761cdc92ea7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/98966d44ea88477b94215761cdc92ea7"}]}] Jul 11 14:45:56.599587 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1829929608 Jul 11 14:45:56.599587 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: 98966d44ea88477b94215761cdc92ea7 Jul 11 14:45:56.599587 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1829929608 Jul 11 14:45:56.599587 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-ServerRescueNegativeTestJSON-1829929608-project Jul 11 14:45:56.599587 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:45:56.599587 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:45:56.599587 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: fa478fb4b41e49c5b1f9291f925cca23 Jul 11 14:45:56.599587 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1829929608-project Jul 11 14:45:56.599587 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:45:56.599587 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance cc0591e3-09b9-41cd-b222-f730282f8572 could not be found. Jul 11 14:45:56.599587 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': 'cc0591e3-09b9-41cd-b222-f730282f8572'} Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cc0591e3-09b9-41cd-b222-f730282f8572 could not be found. Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:45:56.601454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:45:56.603884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:45:56.603884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:45:56.603884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:45:56.603884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:45:56.603884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:45:56.603884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:45:56.603884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:45:56.603884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cc0591e3-09b9-41cd-b222-f730282f8572 could not be found. Jul 11 14:45:56.603884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:45:56.603884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:45:56.603884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:45:56.603884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:45:56.603884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:45:56.603884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:45:56.603884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:45:56.603884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:45:56.603884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cc0591e3-09b9-41cd-b222-f730282f8572 could not be found. Jul 11 14:45:56.603884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:45:56.605971 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a2ff2b48-862c-41b6-8cc0-3cdfdc7d5e94 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Returning 404 to user: Instance cc0591e3-09b9-41cd-b222-f730282f8572 could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:45:56.605971 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a2ff2b48-862c-41b6-8cc0-3cdfdc7d5e94 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572" status: 404 len: 111 microversion: 2.1 time: 0.079380 Jul 11 14:45:56.605971 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 392/804] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:56 2022] GET /compute/v2.1/servers/cc0591e3-09b9-41cd-b222-f730282f8572 => generated 111 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:45:56.618878 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-fcb78cfa-4ad3-4dba-851d-a5eb04f4352c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:56.619843 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-fcb78cfa-4ad3-4dba-851d-a5eb04f4352c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:56.629042 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fcb78cfa-4ad3-4dba-851d-a5eb04f4352c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:56.630409 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fcb78cfa-4ad3-4dba-851d-a5eb04f4352c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:56.631882 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fcb78cfa-4ad3-4dba-851d-a5eb04f4352c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:56.632735 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fcb78cfa-4ad3-4dba-851d-a5eb04f4352c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:56.682044 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-fcb78cfa-4ad3-4dba-851d-a5eb04f4352c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Going to try to terminate instance {{(pid=130159) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:45:56.786152 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-fcb78cfa-4ad3-4dba-851d-a5eb04f4352c tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "DELETE /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 204 len: 0 microversion: 2.1 time: 0.174606 Jul 11 14:45:56.786962 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 413/805] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:45:56 2022] DELETE /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 11 14:45:56.796286 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7d3441f6-a890-4417-864b-9509a753d6c8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:56.797069 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-7d3441f6-a890-4417-864b-9509a753d6c8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:56.804578 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7d3441f6-a890-4417-864b-9509a753d6c8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:56.805031 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7d3441f6-a890-4417-864b-9509a753d6c8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:56.805834 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7d3441f6-a890-4417-864b-9509a753d6c8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:56.806268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7d3441f6-a890-4417-864b-9509a753d6c8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:57.001151 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-7d3441f6-a890-4417-864b-9509a753d6c8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:57.017517 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-7d3441f6-a890-4417-864b-9509a753d6c8 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.224453 Jul 11 14:45:57.018764 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 393/806] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:56 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:57.303121 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-941b66c2-766e-46d7-99b2-e4b64d186aa8 req-cc54a8d3-6e4f-442a-81ed-61546ee06742 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "01762198-189f-4f97-84e5-4e0073a3879b", "tag": "c6732da2-2010-4021-a69d-771c91c0bcac"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:45:57.316122 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-941b66c2-766e-46d7-99b2-e4b64d186aa8 req-cc54a8d3-6e4f-442a-81ed-61546ee06742 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:57.316824 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-941b66c2-766e-46d7-99b2-e4b64d186aa8 req-cc54a8d3-6e4f-442a-81ed-61546ee06742 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:57.338179 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-941b66c2-766e-46d7-99b2-e4b64d186aa8 req-cc54a8d3-6e4f-442a-81ed-61546ee06742 service nova] Creating event network-changed:c6732da2-2010-4021-a69d-771c91c0bcac for instance 01762198-189f-4f97-84e5-4e0073a3879b on hv-cinder-maste Jul 11 14:45:57.345260 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-941b66c2-766e-46d7-99b2-e4b64d186aa8 req-cc54a8d3-6e4f-442a-81ed-61546ee06742 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048739 Jul 11 14:45:57.346744 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 414/807] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:45:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:45:57.529494 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2250cf1b-244d-453c-9320-f96997ed6a4c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:57.530283 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2250cf1b-244d-453c-9320-f96997ed6a4c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:57.537522 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2250cf1b-244d-453c-9320-f96997ed6a4c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:57.538307 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2250cf1b-244d-453c-9320-f96997ed6a4c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:57.539014 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2250cf1b-244d-453c-9320-f96997ed6a4c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:57.539562 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2250cf1b-244d-453c-9320-f96997ed6a4c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:57.685598 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2250cf1b-244d-453c-9320-f96997ed6a4c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:57.694718 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2250cf1b-244d-453c-9320-f96997ed6a4c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 200 len: 1368 microversion: 2.1 time: 0.169263 Jul 11 14:45:57.695522 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 394/808] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:57 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 1368 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:58.045217 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-79d6e676-30a4-4dc1-9112-9ac23eb9d5dc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:58.045990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-79d6e676-30a4-4dc1-9112-9ac23eb9d5dc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:58.052879 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-79d6e676-30a4-4dc1-9112-9ac23eb9d5dc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:58.053329 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-79d6e676-30a4-4dc1-9112-9ac23eb9d5dc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:58.054101 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-79d6e676-30a4-4dc1-9112-9ac23eb9d5dc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:58.054570 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-79d6e676-30a4-4dc1-9112-9ac23eb9d5dc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:58.200562 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-79d6e676-30a4-4dc1-9112-9ac23eb9d5dc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:58.209308 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-79d6e676-30a4-4dc1-9112-9ac23eb9d5dc tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.168506 Jul 11 14:45:58.209900 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 415/809] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:58 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:58.712171 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a99ae72d-9113-4cbd-8003-b66c40baf3ad tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:58.713173 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a99ae72d-9113-4cbd-8003-b66c40baf3ad tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:58.721260 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a99ae72d-9113-4cbd-8003-b66c40baf3ad tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:58.721714 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a99ae72d-9113-4cbd-8003-b66c40baf3ad tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:58.722540 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a99ae72d-9113-4cbd-8003-b66c40baf3ad tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:58.722980 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a99ae72d-9113-4cbd-8003-b66c40baf3ad tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:58.892953 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a99ae72d-9113-4cbd-8003-b66c40baf3ad tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:58.897654 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=100 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:45:58.909573 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a99ae72d-9113-4cbd-8003-b66c40baf3ad tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 200 len: 1368 microversion: 2.1 time: 0.200967 Jul 11 14:45:58.910635 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 395/810] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:58 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 1368 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:59.226737 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e0d00cce-8d18-4550-8612-26da34d6e173 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:59.227413 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e0d00cce-8d18-4550-8612-26da34d6e173 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:59.234918 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e0d00cce-8d18-4550-8612-26da34d6e173 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:59.235428 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e0d00cce-8d18-4550-8612-26da34d6e173 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:59.236340 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e0d00cce-8d18-4550-8612-26da34d6e173 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:59.236852 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e0d00cce-8d18-4550-8612-26da34d6e173 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:59.401848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e0d00cce-8d18-4550-8612-26da34d6e173 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:45:59.409980 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e0d00cce-8d18-4550-8612-26da34d6e173 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.187299 Jul 11 14:45:59.411113 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 416/811] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:59 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:45:59.935421 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-6200f822-0de7-401b-8172-94313dd3713a tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:45:59.936247 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-6200f822-0de7-401b-8172-94313dd3713a tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:45:59.943891 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6200f822-0de7-401b-8172-94313dd3713a tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:59.944350 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6200f822-0de7-401b-8172-94313dd3713a tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:45:59.945335 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6200f822-0de7-401b-8172-94313dd3713a tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:45:59.945816 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6200f822-0de7-401b-8172-94313dd3713a tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:00.182581 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6200f822-0de7-401b-8172-94313dd3713a tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:00.197734 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-6200f822-0de7-401b-8172-94313dd3713a tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 200 len: 1368 microversion: 2.1 time: 0.266363 Jul 11 14:46:00.199318 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 396/812] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:45:59 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 1368 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:46:00.429787 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-dadfe5f5-6f0a-43e3-af65-696599e26ef5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:00.430703 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-dadfe5f5-6f0a-43e3-af65-696599e26ef5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:00.440445 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-dadfe5f5-6f0a-43e3-af65-696599e26ef5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:00.440997 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-dadfe5f5-6f0a-43e3-af65-696599e26ef5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:00.442464 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-dadfe5f5-6f0a-43e3-af65-696599e26ef5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:00.442977 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-dadfe5f5-6f0a-43e3-af65-696599e26ef5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:00.771733 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-c6972205-f79d-49d7-9624-bee569a1cf5c req-fe5cfe9d-5864-416b-8fbb-eac2b8e0394e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a0498735-9e0a-4f14-990d-a6d5076e742a", "name": "network-vif-deleted", "tag": "27aef129-6bd8-4686-9e6d-915b57154c5a"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:46:00.772367 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-dadfe5f5-6f0a-43e3-af65-696599e26ef5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:00.783700 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-dadfe5f5-6f0a-43e3-af65-696599e26ef5 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1521 microversion: 2.1 time: 0.359120 Jul 11 14:46:00.797120 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-c6972205-f79d-49d7-9624-bee569a1cf5c req-fe5cfe9d-5864-416b-8fbb-eac2b8e0394e service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:00.797938 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-c6972205-f79d-49d7-9624-bee569a1cf5c req-fe5cfe9d-5864-416b-8fbb-eac2b8e0394e service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:00.799012 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 417/813] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:46:00 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1521 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:46:00.830734 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-c6972205-f79d-49d7-9624-bee569a1cf5c req-fe5cfe9d-5864-416b-8fbb-eac2b8e0394e service nova] Creating event network-vif-deleted:27aef129-6bd8-4686-9e6d-915b57154c5a for instance a0498735-9e0a-4f14-990d-a6d5076e742a on hv-cinder-maste Jul 11 14:46:00.837865 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-c6972205-f79d-49d7-9624-bee569a1cf5c req-fe5cfe9d-5864-416b-8fbb-eac2b8e0394e service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.073201 Jul 11 14:46:00.844683 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 397/814] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:46:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:46:01.226360 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2c46067f-d181-46e1-ae66-0dd1a6422401 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:01.227495 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2c46067f-d181-46e1-ae66-0dd1a6422401 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:01.234940 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2c46067f-d181-46e1-ae66-0dd1a6422401 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:01.235388 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2c46067f-d181-46e1-ae66-0dd1a6422401 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:01.236167 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2c46067f-d181-46e1-ae66-0dd1a6422401 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:01.236855 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2c46067f-d181-46e1-ae66-0dd1a6422401 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:01.410593 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-2c46067f-d181-46e1-ae66-0dd1a6422401 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:01.421033 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2c46067f-d181-46e1-ae66-0dd1a6422401 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 200 len: 1368 microversion: 2.1 time: 0.199612 Jul 11 14:46:01.421955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 418/815] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:46:01 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 1368 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:46:01.807483 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-cc85e219-524d-47d6-aa6c-8054a789ff79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:01.808586 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-cc85e219-524d-47d6-aa6c-8054a789ff79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:01.815861 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc85e219-524d-47d6-aa6c-8054a789ff79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:01.816481 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc85e219-524d-47d6-aa6c-8054a789ff79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:01.817711 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc85e219-524d-47d6-aa6c-8054a789ff79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:01.818287 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc85e219-524d-47d6-aa6c-8054a789ff79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:01.854525 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc85e219-524d-47d6-aa6c-8054a789ff79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:01.854983 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc85e219-524d-47d6-aa6c-8054a789ff79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:01.855439 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.objects.instance [None req-cc85e219-524d-47d6-aa6c-8054a789ff79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lazy-loading 'fault' on Instance uuid a0498735-9e0a-4f14-990d-a6d5076e742a {{(pid=130160) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:46:01.909693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-cc85e219-524d-47d6-aa6c-8054a789ff79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fa478fb4b41e49c5b1f9291f925cca23', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '98966d44ea88477b94215761cdc92ea7', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:01.918884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-cc85e219-524d-47d6-aa6c-8054a789ff79 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 200 len: 1312 microversion: 2.1 time: 0.116362 Jul 11 14:46:01.919661 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 398/816] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:46:01 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 1312 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:46:02.168227 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-990e6f78-abef-4f1a-9702-45b57751e7fc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:02.168834 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-990e6f78-abef-4f1a-9702-45b57751e7fc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:02.175853 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-990e6f78-abef-4f1a-9702-45b57751e7fc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:02.176420 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-990e6f78-abef-4f1a-9702-45b57751e7fc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:02.177361 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-990e6f78-abef-4f1a-9702-45b57751e7fc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:02.177883 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-990e6f78-abef-4f1a-9702-45b57751e7fc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:02.205955 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-990e6f78-abef-4f1a-9702-45b57751e7fc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad/ips" status: 200 len: 151 microversion: 2.20 time: 0.107745 Jul 11 14:46:02.206539 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 419/817] 10.222.0.72 () {60 vars in 1288 bytes} [Mon Jul 11 14:46:02 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad/ips => generated 151 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Jul 11 14:46:02.446950 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f99d0169-41fa-481f-857e-e0d41c4a41b0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:02.447648 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f99d0169-41fa-481f-857e-e0d41c4a41b0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:02.456320 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f99d0169-41fa-481f-857e-e0d41c4a41b0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:02.459006 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f99d0169-41fa-481f-857e-e0d41c4a41b0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:02.460016 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f99d0169-41fa-481f-857e-e0d41c4a41b0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:02.460744 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f99d0169-41fa-481f-857e-e0d41c4a41b0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:02.683654 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f99d0169-41fa-481f-857e-e0d41c4a41b0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:02.692878 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f99d0169-41fa-481f-857e-e0d41c4a41b0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 200 len: 1368 microversion: 2.1 time: 0.252335 Jul 11 14:46:02.693720 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 399/818] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:46:02 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 1368 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:46:02.938936 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-da81292d-9cd7-473a-a25f-68c0e2747e02 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:02.939816 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-da81292d-9cd7-473a-a25f-68c0e2747e02 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] [instance: a0498735-9e0a-4f14-990d-a6d5076e742a] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:02.947783 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-da81292d-9cd7-473a-a25f-68c0e2747e02 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:02.948273 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-da81292d-9cd7-473a-a25f-68c0e2747e02 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:02.949314 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-da81292d-9cd7-473a-a25f-68c0e2747e02 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:02.949606 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-da81292d-9cd7-473a-a25f-68c0e2747e02 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:02.966704 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-da81292d-9cd7-473a-a25f-68c0e2747e02 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] HTTP exception thrown: Instance a0498735-9e0a-4f14-990d-a6d5076e742a could not be found. Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-da81292d-9cd7-473a-a25f-68c0e2747e02 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Request method failure captured: Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a HTTP/1.1 Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZVO0vcEhdpPwwfJZnu1y3p7Tg0Q8GENrUVh0NohR9oAPNIjKFLY8v62KxMOiNHGfwF-EVGge1hPwlyTCX1uExRac1NToAISnaSDmVmpVWA-TKGS1rzgSZVkfiPxYTNCyZmEJBrMLK9NH_pMk0kTrm35o7KBc7uMvJqd_pTjsh0TKE Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: 98966d44ea88477b94215761cdc92ea7 Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1829929608 Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:46:02.969147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:46:02.971518 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/98966d44ea88477b94215761cdc92ea7"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/98966d44ea88477b94215761cdc92ea7"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_98966d44ea88477b94215761cdc92ea7"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/98966d44ea88477b94215761cdc92ea7"}]}] Jul 11 14:46:02.971518 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1829929608 Jul 11 14:46:02.971518 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: 98966d44ea88477b94215761cdc92ea7 Jul 11 14:46:02.971518 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1829929608 Jul 11 14:46:02.971518 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-ServerRescueNegativeTestJSON-1829929608-project Jul 11 14:46:02.971518 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:46:02.971518 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:46:02.971518 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: fa478fb4b41e49c5b1f9291f925cca23 Jul 11 14:46:02.971518 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1829929608-project Jul 11 14:46:02.971518 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:46:02.971518 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance a0498735-9e0a-4f14-990d-a6d5076e742a could not be found. Jul 11 14:46:02.971518 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': 'a0498735-9e0a-4f14-990d-a6d5076e742a'} Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a0498735-9e0a-4f14-990d-a6d5076e742a could not be found. Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:46:02.972973 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:46:02.975905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:46:02.975905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:46:02.975905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:46:02.975905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:46:02.975905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:46:02.975905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:46:02.975905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:46:02.975905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a0498735-9e0a-4f14-990d-a6d5076e742a could not be found. Jul 11 14:46:02.975905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:46:02.975905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:46:02.975905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:46:02.975905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:46:02.975905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:46:02.975905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:46:02.975905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:46:02.975905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:46:02.975905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a0498735-9e0a-4f14-990d-a6d5076e742a could not be found. Jul 11 14:46:02.975905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:46:02.978063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-da81292d-9cd7-473a-a25f-68c0e2747e02 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] Returning 404 to user: Instance a0498735-9e0a-4f14-990d-a6d5076e742a could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:46:02.978063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-da81292d-9cd7-473a-a25f-68c0e2747e02 tempest-ServerRescueNegativeTestJSON-1829929608 tempest-ServerRescueNegativeTestJSON-1829929608-project] 10.222.0.72 "GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a" status: 404 len: 111 microversion: 2.1 time: 0.037309 Jul 11 14:46:02.978063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 420/819] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:46:02 2022] GET /compute/v2.1/servers/a0498735-9e0a-4f14-990d-a6d5076e742a => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:46:03.716273 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-ecea0bfd-40a9-4081-8aad-7bae18fc4fb7 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:03.717472 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-ecea0bfd-40a9-4081-8aad-7bae18fc4fb7 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:03.729778 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ecea0bfd-40a9-4081-8aad-7bae18fc4fb7 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:03.730612 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ecea0bfd-40a9-4081-8aad-7bae18fc4fb7 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:03.732095 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ecea0bfd-40a9-4081-8aad-7bae18fc4fb7 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:03.732884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ecea0bfd-40a9-4081-8aad-7bae18fc4fb7 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:03.875724 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=2 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:46:03.959120 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-ecea0bfd-40a9-4081-8aad-7bae18fc4fb7 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:03.968577 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-ecea0bfd-40a9-4081-8aad-7bae18fc4fb7 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 200 len: 1368 microversion: 2.1 time: 0.259325 Jul 11 14:46:03.969851 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 400/820] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:46:03 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 1368 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:46:04.751239 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a1155cef-c40c-4349-b816-261868101944 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:46:04.752607 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a1155cef-c40c-4349-b816-261868101944 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:04.759948 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=2 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:46:04.761453 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=10 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:46:04.766878 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a1155cef-c40c-4349-b816-261868101944 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:04.768155 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a1155cef-c40c-4349-b816-261868101944 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:04.769246 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a1155cef-c40c-4349-b816-261868101944 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:04.770457 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a1155cef-c40c-4349-b816-261868101944 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:04.849844 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:46:04.854892 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.objects.instance [None req-a1155cef-c40c-4349-b816-261868101944 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lazy-loading 'flavor' on Instance uuid 43772fe7-b391-4878-bbea-658602d822ad {{(pid=130159) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:46:04.999225 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-b4404403-2c83-40c7-b0fb-48b7c9b71d39 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:05.000292 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-b4404403-2c83-40c7-b0fb-48b7c9b71d39 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:05.014627 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b4404403-2c83-40c7-b0fb-48b7c9b71d39 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:05.015815 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b4404403-2c83-40c7-b0fb-48b7c9b71d39 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:05.017393 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b4404403-2c83-40c7-b0fb-48b7c9b71d39 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:05.018101 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b4404403-2c83-40c7-b0fb-48b7c9b71d39 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:05.172898 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a1155cef-c40c-4349-b816-261868101944 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "POST /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad/action" status: 202 len: 0 microversion: 2.20 time: 0.425741 Jul 11 14:46:05.174716 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 421/821] 10.222.0.72 () {62 vars in 1318 bytes} [Mon Jul 11 14:46:04 2022] POST /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad/action => generated 0 bytes in 429 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Jul 11 14:46:05.197598 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-27edfef6-c0bb-4b64-b314-2c75da417e75 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:05.198369 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-27edfef6-c0bb-4b64-b314-2c75da417e75 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:05.205818 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-27edfef6-c0bb-4b64-b314-2c75da417e75 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:05.206270 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-27edfef6-c0bb-4b64-b314-2c75da417e75 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:05.207392 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-27edfef6-c0bb-4b64-b314-2c75da417e75 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:05.207836 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-27edfef6-c0bb-4b64-b314-2c75da417e75 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:05.270384 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-b4404403-2c83-40c7-b0fb-48b7c9b71d39 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:05.282202 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-b4404403-2c83-40c7-b0fb-48b7c9b71d39 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 200 len: 1368 microversion: 2.1 time: 0.290483 Jul 11 14:46:05.283229 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 401/822] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:46:04 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 1368 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:46:05.516512 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-27edfef6-c0bb-4b64-b314-2c75da417e75 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:05.533645 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-27edfef6-c0bb-4b64-b314-2c75da417e75 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:05.537875 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-27edfef6-c0bb-4b64-b314-2c75da417e75 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:05.540586 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-27edfef6-c0bb-4b64-b314-2c75da417e75 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1722 microversion: 2.20 time: 0.350677 Jul 11 14:46:05.542445 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 422/823] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:05 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1722 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:06.304956 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-046bd01e-265b-43cd-84d5-f992b57fada9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:06.307240 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-046bd01e-265b-43cd-84d5-f992b57fada9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:06.322193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-046bd01e-265b-43cd-84d5-f992b57fada9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:06.323170 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-046bd01e-265b-43cd-84d5-f992b57fada9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:06.324562 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-046bd01e-265b-43cd-84d5-f992b57fada9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:06.325561 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-046bd01e-265b-43cd-84d5-f992b57fada9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:06.395329 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-72b33127-67c8-4836-a332-8e54b8e4a490 req-7c9a6944-5905-4f53-9ae5-98cbb3c19961 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "01762198-189f-4f97-84e5-4e0073a3879b", "name": "network-vif-plugged", "status": "completed", "tag": "c6732da2-2010-4021-a69d-771c91c0bcac"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:46:06.404570 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-72b33127-67c8-4836-a332-8e54b8e4a490 req-7c9a6944-5905-4f53-9ae5-98cbb3c19961 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:06.405041 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-72b33127-67c8-4836-a332-8e54b8e4a490 req-7c9a6944-5905-4f53-9ae5-98cbb3c19961 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:06.417863 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-72b33127-67c8-4836-a332-8e54b8e4a490 req-7c9a6944-5905-4f53-9ae5-98cbb3c19961 service nova] Creating event network-vif-plugged:c6732da2-2010-4021-a69d-771c91c0bcac for instance 01762198-189f-4f97-84e5-4e0073a3879b on hv-cinder-maste Jul 11 14:46:06.423380 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-72b33127-67c8-4836-a332-8e54b8e4a490 req-7c9a6944-5905-4f53-9ae5-98cbb3c19961 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.158986 Jul 11 14:46:06.424002 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 402/824] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:46:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:46:06.550097 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-046bd01e-265b-43cd-84d5-f992b57fada9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:06.579098 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-227803e7-4aaa-4996-baef-9198abbbf6ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:06.581419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-227803e7-4aaa-4996-baef-9198abbbf6ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:06.593869 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-227803e7-4aaa-4996-baef-9198abbbf6ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:06.595331 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-227803e7-4aaa-4996-baef-9198abbbf6ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:06.596396 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-227803e7-4aaa-4996-baef-9198abbbf6ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:06.596878 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-227803e7-4aaa-4996-baef-9198abbbf6ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:06.600705 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-046bd01e-265b-43cd-84d5-f992b57fada9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 200 len: 1368 microversion: 2.1 time: 0.296999 Jul 11 14:46:06.616628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 423/825] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:46:06 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 1368 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:46:07.143316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-227803e7-4aaa-4996-baef-9198abbbf6ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:07.157145 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-227803e7-4aaa-4996-baef-9198abbbf6ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:07.159867 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-227803e7-4aaa-4996-baef-9198abbbf6ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:07.162178 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-227803e7-4aaa-4996-baef-9198abbbf6ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1722 microversion: 2.20 time: 0.595562 Jul 11 14:46:07.163430 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 403/826] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:06 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1722 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:07.633986 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-99a001d8-d305-41b2-8455-0f19476b6622 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:07.635367 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-99a001d8-d305-41b2-8455-0f19476b6622 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:07.649021 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-99a001d8-d305-41b2-8455-0f19476b6622 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:07.650106 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-99a001d8-d305-41b2-8455-0f19476b6622 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:07.651690 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-99a001d8-d305-41b2-8455-0f19476b6622 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:07.652698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-99a001d8-d305-41b2-8455-0f19476b6622 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:08.083127 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-99a001d8-d305-41b2-8455-0f19476b6622 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:08.095907 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-99a001d8-d305-41b2-8455-0f19476b6622 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 200 len: 1368 microversion: 2.1 time: 0.470289 Jul 11 14:46:08.096764 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 424/827] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:46:07 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 1368 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:46:08.194334 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-6f47d560-7e43-4887-88fb-afc5e9b901bd tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:08.195555 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-6f47d560-7e43-4887-88fb-afc5e9b901bd tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:08.206470 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6f47d560-7e43-4887-88fb-afc5e9b901bd tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:08.207807 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6f47d560-7e43-4887-88fb-afc5e9b901bd tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:08.208751 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6f47d560-7e43-4887-88fb-afc5e9b901bd tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:08.209408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6f47d560-7e43-4887-88fb-afc5e9b901bd tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:08.378784 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6f47d560-7e43-4887-88fb-afc5e9b901bd tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:08.392190 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6f47d560-7e43-4887-88fb-afc5e9b901bd tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:08.396106 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6f47d560-7e43-4887-88fb-afc5e9b901bd tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:08.398413 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-6f47d560-7e43-4887-88fb-afc5e9b901bd tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1722 microversion: 2.20 time: 0.214521 Jul 11 14:46:08.399537 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 404/828] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:08 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1722 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:09.115889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3bb352f3-1f9c-4059-bb71-bbe8fc5156d9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:09.116900 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3bb352f3-1f9c-4059-bb71-bbe8fc5156d9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:09.123423 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3bb352f3-1f9c-4059-bb71-bbe8fc5156d9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:09.123869 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3bb352f3-1f9c-4059-bb71-bbe8fc5156d9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:09.124651 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3bb352f3-1f9c-4059-bb71-bbe8fc5156d9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:09.125097 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3bb352f3-1f9c-4059-bb71-bbe8fc5156d9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:09.301318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3bb352f3-1f9c-4059-bb71-bbe8fc5156d9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:09.317376 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3bb352f3-1f9c-4059-bb71-bbe8fc5156d9 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 200 len: 1557 microversion: 2.1 time: 0.205144 Jul 11 14:46:09.318163 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 425/829] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:46:09 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 1557 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:46:09.337323 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-99042ca7-28e2-4b53-828c-6e8effdf7955 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:09.338164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-99042ca7-28e2-4b53-828c-6e8effdf7955 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:09.340894 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=89,nova_api:UPDATE=1,nova_api:DELETE=2 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:46:09.349608 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-99042ca7-28e2-4b53-828c-6e8effdf7955 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:09.350259 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-99042ca7-28e2-4b53-828c-6e8effdf7955 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:09.351489 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-99042ca7-28e2-4b53-828c-6e8effdf7955 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:09.352158 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-99042ca7-28e2-4b53-828c-6e8effdf7955 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:09.427612 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-be4d1bb9-4c4d-4474-b207-fdc5e382675c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:09.428240 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-be4d1bb9-4c4d-4474-b207-fdc5e382675c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:09.435766 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-be4d1bb9-4c4d-4474-b207-fdc5e382675c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:09.436265 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-be4d1bb9-4c4d-4474-b207-fdc5e382675c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:09.437132 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-be4d1bb9-4c4d-4474-b207-fdc5e382675c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:09.437596 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-be4d1bb9-4c4d-4474-b207-fdc5e382675c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:09.450751 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-99042ca7-28e2-4b53-828c-6e8effdf7955 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.118700 Jul 11 14:46:09.451702 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 405/830] 10.222.0.72 () {58 vars in 1274 bytes} [Mon Jul 11 14:46:09 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b/os-interface => generated 284 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:46:09.768470 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-be4d1bb9-4c4d-4474-b207-fdc5e382675c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:09.777964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-be4d1bb9-4c4d-4474-b207-fdc5e382675c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:09.779535 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-be4d1bb9-4c4d-4474-b207-fdc5e382675c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:09.780992 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-be4d1bb9-4c4d-4474-b207-fdc5e382675c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1722 microversion: 2.20 time: 0.357988 Jul 11 14:46:09.781725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 426/831] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:09 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1722 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:46:10.805105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-4e0cc8a1-3037-4215-8bd0-9b3e972b1ae3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:10.807363 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-4e0cc8a1-3037-4215-8bd0-9b3e972b1ae3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:10.818834 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4e0cc8a1-3037-4215-8bd0-9b3e972b1ae3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:10.819647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4e0cc8a1-3037-4215-8bd0-9b3e972b1ae3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:10.821433 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4e0cc8a1-3037-4215-8bd0-9b3e972b1ae3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:10.822749 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4e0cc8a1-3037-4215-8bd0-9b3e972b1ae3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:11.068279 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4e0cc8a1-3037-4215-8bd0-9b3e972b1ae3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:11.080458 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4e0cc8a1-3037-4215-8bd0-9b3e972b1ae3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:11.084159 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4e0cc8a1-3037-4215-8bd0-9b3e972b1ae3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:11.086751 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-4e0cc8a1-3037-4215-8bd0-9b3e972b1ae3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1722 microversion: 2.20 time: 0.289362 Jul 11 14:46:11.088577 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 406/832] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:10 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1722 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:11.110789 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-cdc942c6-85ab-42ec-a26c-a335a21f0ecb req-e3fd2980-aee3-485f-95ed-4e07d6608407 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "01762198-189f-4f97-84e5-4e0073a3879b", "tag": "c6732da2-2010-4021-a69d-771c91c0bcac"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:46:11.126582 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-cdc942c6-85ab-42ec-a26c-a335a21f0ecb req-e3fd2980-aee3-485f-95ed-4e07d6608407 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:11.127277 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-cdc942c6-85ab-42ec-a26c-a335a21f0ecb req-e3fd2980-aee3-485f-95ed-4e07d6608407 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:11.144256 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-cdc942c6-85ab-42ec-a26c-a335a21f0ecb req-e3fd2980-aee3-485f-95ed-4e07d6608407 service nova] Creating event network-changed:c6732da2-2010-4021-a69d-771c91c0bcac for instance 01762198-189f-4f97-84e5-4e0073a3879b on hv-cinder-maste Jul 11 14:46:11.151824 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-cdc942c6-85ab-42ec-a26c-a335a21f0ecb req-e3fd2980-aee3-485f-95ed-4e07d6608407 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048942 Jul 11 14:46:11.152980 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 427/833] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:46:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:46:12.115807 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a5164ecd-e4ca-4c39-b3ed-3bb288689420 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:12.117777 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a5164ecd-e4ca-4c39-b3ed-3bb288689420 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:12.130817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a5164ecd-e4ca-4c39-b3ed-3bb288689420 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:12.132095 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a5164ecd-e4ca-4c39-b3ed-3bb288689420 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:12.133844 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a5164ecd-e4ca-4c39-b3ed-3bb288689420 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:12.134658 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a5164ecd-e4ca-4c39-b3ed-3bb288689420 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:12.382582 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a5164ecd-e4ca-4c39-b3ed-3bb288689420 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:12.399998 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a5164ecd-e4ca-4c39-b3ed-3bb288689420 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:12.403568 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a5164ecd-e4ca-4c39-b3ed-3bb288689420 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:12.405844 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a5164ecd-e4ca-4c39-b3ed-3bb288689420 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1722 microversion: 2.20 time: 0.296818 Jul 11 14:46:12.406906 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 407/834] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:12 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1722 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:46:13.439740 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-7bf51f57-89da-4aa3-9a5f-ae8c59d39ba6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:13.440703 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-7bf51f57-89da-4aa3-9a5f-ae8c59d39ba6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:13.452319 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7bf51f57-89da-4aa3-9a5f-ae8c59d39ba6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:13.453016 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7bf51f57-89da-4aa3-9a5f-ae8c59d39ba6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:13.454402 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7bf51f57-89da-4aa3-9a5f-ae8c59d39ba6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:13.455230 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7bf51f57-89da-4aa3-9a5f-ae8c59d39ba6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:13.641252 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7bf51f57-89da-4aa3-9a5f-ae8c59d39ba6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:13.650484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7bf51f57-89da-4aa3-9a5f-ae8c59d39ba6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:13.652096 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7bf51f57-89da-4aa3-9a5f-ae8c59d39ba6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:13.653339 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-7bf51f57-89da-4aa3-9a5f-ae8c59d39ba6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1722 microversion: 2.20 time: 0.223366 Jul 11 14:46:13.653953 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 428/835] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:13 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1722 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:14.682160 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f6d89b84-0f6e-4515-b76e-0972d9a7d1ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:14.682845 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f6d89b84-0f6e-4515-b76e-0972d9a7d1ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:14.692642 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f6d89b84-0f6e-4515-b76e-0972d9a7d1ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:14.693481 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f6d89b84-0f6e-4515-b76e-0972d9a7d1ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:14.694990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f6d89b84-0f6e-4515-b76e-0972d9a7d1ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:14.695793 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f6d89b84-0f6e-4515-b76e-0972d9a7d1ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:14.920913 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f6d89b84-0f6e-4515-b76e-0972d9a7d1ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:14.931807 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f6d89b84-0f6e-4515-b76e-0972d9a7d1ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:14.933341 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f6d89b84-0f6e-4515-b76e-0972d9a7d1ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:14.934640 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f6d89b84-0f6e-4515-b76e-0972d9a7d1ec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1722 microversion: 2.20 time: 0.256803 Jul 11 14:46:14.935235 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 408/836] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:14 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1722 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:15.964285 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-7bc88fa2-7052-4787-94f5-82007e741b5a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:15.965228 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-7bc88fa2-7052-4787-94f5-82007e741b5a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:15.973787 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7bc88fa2-7052-4787-94f5-82007e741b5a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:15.974247 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7bc88fa2-7052-4787-94f5-82007e741b5a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:15.975100 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7bc88fa2-7052-4787-94f5-82007e741b5a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:15.975538 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7bc88fa2-7052-4787-94f5-82007e741b5a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:16.126137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7bc88fa2-7052-4787-94f5-82007e741b5a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:16.136414 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7bc88fa2-7052-4787-94f5-82007e741b5a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:16.138341 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7bc88fa2-7052-4787-94f5-82007e741b5a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:16.139674 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-7bc88fa2-7052-4787-94f5-82007e741b5a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1722 microversion: 2.20 time: 0.180148 Jul 11 14:46:16.140374 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 429/837] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:15 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1722 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:46:17.159604 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-1c6d17fa-5798-4918-b85f-87828c0b218c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:17.160223 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-1c6d17fa-5798-4918-b85f-87828c0b218c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:17.168087 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1c6d17fa-5798-4918-b85f-87828c0b218c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:17.168548 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1c6d17fa-5798-4918-b85f-87828c0b218c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:17.169384 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1c6d17fa-5798-4918-b85f-87828c0b218c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:17.169831 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1c6d17fa-5798-4918-b85f-87828c0b218c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:17.315309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-1c6d17fa-5798-4918-b85f-87828c0b218c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:17.330410 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-1c6d17fa-5798-4918-b85f-87828c0b218c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:17.334828 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-1c6d17fa-5798-4918-b85f-87828c0b218c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:17.336871 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-1c6d17fa-5798-4918-b85f-87828c0b218c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1722 microversion: 2.20 time: 0.181241 Jul 11 14:46:17.337794 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 409/838] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:17 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1722 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:46:18.357035 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e94388f9-42ba-48c7-a36a-d6e792187161 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:18.357863 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e94388f9-42ba-48c7-a36a-d6e792187161 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:18.365697 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e94388f9-42ba-48c7-a36a-d6e792187161 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:18.366597 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e94388f9-42ba-48c7-a36a-d6e792187161 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:18.367517 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e94388f9-42ba-48c7-a36a-d6e792187161 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:18.368046 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e94388f9-42ba-48c7-a36a-d6e792187161 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:18.603803 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e94388f9-42ba-48c7-a36a-d6e792187161 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:18.622172 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e94388f9-42ba-48c7-a36a-d6e792187161 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:18.625812 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e94388f9-42ba-48c7-a36a-d6e792187161 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:18.628109 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e94388f9-42ba-48c7-a36a-d6e792187161 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1722 microversion: 2.20 time: 0.274735 Jul 11 14:46:18.629743 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 430/839] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:18 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1722 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:46:19.657018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-828795c7-f42c-4311-a805-3b1776682dee tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:19.658175 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-828795c7-f42c-4311-a805-3b1776682dee tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:19.672363 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-828795c7-f42c-4311-a805-3b1776682dee tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:19.673101 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-828795c7-f42c-4311-a805-3b1776682dee tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:19.674072 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-828795c7-f42c-4311-a805-3b1776682dee tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:19.674880 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-828795c7-f42c-4311-a805-3b1776682dee tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:19.877882 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-828795c7-f42c-4311-a805-3b1776682dee tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:19.891638 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-828795c7-f42c-4311-a805-3b1776682dee tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:19.893895 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-828795c7-f42c-4311-a805-3b1776682dee tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:19.895848 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-828795c7-f42c-4311-a805-3b1776682dee tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1722 microversion: 2.20 time: 0.247320 Jul 11 14:46:19.896895 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 410/840] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:19 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1722 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:20.917460 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c73d9042-9d3c-4cd7-ac8c-89b9295be6a3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:20.918359 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c73d9042-9d3c-4cd7-ac8c-89b9295be6a3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:20.926206 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c73d9042-9d3c-4cd7-ac8c-89b9295be6a3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:20.926823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c73d9042-9d3c-4cd7-ac8c-89b9295be6a3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:20.927650 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c73d9042-9d3c-4cd7-ac8c-89b9295be6a3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:20.928161 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c73d9042-9d3c-4cd7-ac8c-89b9295be6a3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:21.095094 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c73d9042-9d3c-4cd7-ac8c-89b9295be6a3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:21.109453 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c73d9042-9d3c-4cd7-ac8c-89b9295be6a3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:21.112915 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c73d9042-9d3c-4cd7-ac8c-89b9295be6a3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:21.115061 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c73d9042-9d3c-4cd7-ac8c-89b9295be6a3 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1722 microversion: 2.20 time: 0.202967 Jul 11 14:46:21.115862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 431/841] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:20 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1722 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:22.144959 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8f9260b2-7a5d-4fc8-8e2b-41f3baf2f4e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:22.146241 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-8f9260b2-7a5d-4fc8-8e2b-41f3baf2f4e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:22.158368 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8f9260b2-7a5d-4fc8-8e2b-41f3baf2f4e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:22.159414 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8f9260b2-7a5d-4fc8-8e2b-41f3baf2f4e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:22.161131 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8f9260b2-7a5d-4fc8-8e2b-41f3baf2f4e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:22.162012 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8f9260b2-7a5d-4fc8-8e2b-41f3baf2f4e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:22.386194 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8f9260b2-7a5d-4fc8-8e2b-41f3baf2f4e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:22.394960 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8f9260b2-7a5d-4fc8-8e2b-41f3baf2f4e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:22.397024 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8f9260b2-7a5d-4fc8-8e2b-41f3baf2f4e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:22.398499 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8f9260b2-7a5d-4fc8-8e2b-41f3baf2f4e9 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1722 microversion: 2.20 time: 0.266026 Jul 11 14:46:22.399272 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 411/842] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:22 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1722 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:22.969182 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1fc4369a-4d1d-4c20-999d-d8224f8e9e16 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-1911000918"}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:46:22.970576 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1fc4369a-4d1d-4c20-999d-d8224f8e9e16 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:22.979136 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1fc4369a-4d1d-4c20-999d-d8224f8e9e16 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:22.979664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1fc4369a-4d1d-4c20-999d-d8224f8e9e16 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:22.980517 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1fc4369a-4d1d-4c20-999d-d8224f8e9e16 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:22.981083 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1fc4369a-4d1d-4c20-999d-d8224f8e9e16 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:23.161737 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.network.security_group_api [None req-1fc4369a-4d1d-4c20-999d-d8224f8e9e16 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Removing security group 65fd9fc5-5855-468e-9322-c09a03998b07 from port eacb6063-c62f-4742-8ca6-7f93d25fdaf8 Jul 11 14:46:23.422592 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-5e83b20e-6c9a-4ab8-8ff6-2f4e75dd91bb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:23.423800 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-5e83b20e-6c9a-4ab8-8ff6-2f4e75dd91bb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:23.436653 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5e83b20e-6c9a-4ab8-8ff6-2f4e75dd91bb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:23.437828 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5e83b20e-6c9a-4ab8-8ff6-2f4e75dd91bb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:23.440082 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5e83b20e-6c9a-4ab8-8ff6-2f4e75dd91bb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:23.441067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5e83b20e-6c9a-4ab8-8ff6-2f4e75dd91bb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:23.700821 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5e83b20e-6c9a-4ab8-8ff6-2f4e75dd91bb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:23.716399 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5e83b20e-6c9a-4ab8-8ff6-2f4e75dd91bb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:23.720020 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5e83b20e-6c9a-4ab8-8ff6-2f4e75dd91bb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:23.723650 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-5e83b20e-6c9a-4ab8-8ff6-2f4e75dd91bb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1722 microversion: 2.20 time: 0.307868 Jul 11 14:46:23.724965 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 412/843] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:23 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1722 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:23.831239 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1fc4369a-4d1d-4c20-999d-d8224f8e9e16 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "POST /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/action" status: 202 len: 0 microversion: 2.1 time: 0.866483 Jul 11 14:46:23.835199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 432/844] 10.222.0.72 () {60 vars in 1277 bytes} [Mon Jul 11 14:46:22 2022] POST /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/action => generated 0 bytes in 871 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 11 14:46:23.862476 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-4cc0ba09-aed6-4356-809d-23df509d580c req-d0e985db-2bed-415a-9ba6-1571a09426f5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0033415c-3adf-4fe8-bac6-60cf7774143f", "tag": "eacb6063-c62f-4742-8ca6-7f93d25fdaf8"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:46:23.881774 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-4cc0ba09-aed6-4356-809d-23df509d580c req-d0e985db-2bed-415a-9ba6-1571a09426f5 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:23.886760 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-4cc0ba09-aed6-4356-809d-23df509d580c req-d0e985db-2bed-415a-9ba6-1571a09426f5 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:23.918750 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-4cc0ba09-aed6-4356-809d-23df509d580c req-d0e985db-2bed-415a-9ba6-1571a09426f5 service nova] Creating event network-changed:eacb6063-c62f-4742-8ca6-7f93d25fdaf8 for instance 0033415c-3adf-4fe8-bac6-60cf7774143f on hv-cinder-maste Jul 11 14:46:23.934807 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-4cc0ba09-aed6-4356-809d-23df509d580c req-d0e985db-2bed-415a-9ba6-1571a09426f5 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.069405 Jul 11 14:46:23.935756 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 413/845] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:46:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:46:24.757158 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c00627e9-060d-4abb-a92e-ed57295adf12 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:24.761699 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c00627e9-060d-4abb-a92e-ed57295adf12 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:24.773232 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=51,nova_api:UPDATE=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:46:24.792011 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c00627e9-060d-4abb-a92e-ed57295adf12 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:24.792683 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c00627e9-060d-4abb-a92e-ed57295adf12 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:24.793954 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c00627e9-060d-4abb-a92e-ed57295adf12 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:24.795934 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c00627e9-060d-4abb-a92e-ed57295adf12 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:25.240053 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c00627e9-060d-4abb-a92e-ed57295adf12 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:25.249399 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c00627e9-060d-4abb-a92e-ed57295adf12 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:25.251142 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c00627e9-060d-4abb-a92e-ed57295adf12 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:25.252369 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c00627e9-060d-4abb-a92e-ed57295adf12 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1722 microversion: 2.20 time: 0.500999 Jul 11 14:46:25.253144 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 433/846] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:24 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1722 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:26.286828 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-513b0040-ef94-49d0-8b57-ba841a93891a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:26.300835 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-513b0040-ef94-49d0-8b57-ba841a93891a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:26.315116 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-513b0040-ef94-49d0-8b57-ba841a93891a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:26.316851 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-513b0040-ef94-49d0-8b57-ba841a93891a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:26.319043 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-513b0040-ef94-49d0-8b57-ba841a93891a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:26.320038 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-513b0040-ef94-49d0-8b57-ba841a93891a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:26.366541 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=100 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:46:26.505625 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-aea5c0c8-9a86-49ec-8f2f-2a99fd433d64 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:26.506519 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-aea5c0c8-9a86-49ec-8f2f-2a99fd433d64 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:26.516447 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-aea5c0c8-9a86-49ec-8f2f-2a99fd433d64 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:26.516892 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-aea5c0c8-9a86-49ec-8f2f-2a99fd433d64 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:26.521739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-aea5c0c8-9a86-49ec-8f2f-2a99fd433d64 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:26.522296 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-aea5c0c8-9a86-49ec-8f2f-2a99fd433d64 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:26.577076 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-513b0040-ef94-49d0-8b57-ba841a93891a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:26.605144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-513b0040-ef94-49d0-8b57-ba841a93891a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:26.609657 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-513b0040-ef94-49d0-8b57-ba841a93891a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:26.614261 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-513b0040-ef94-49d0-8b57-ba841a93891a tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1743 microversion: 2.20 time: 0.333881 Jul 11 14:46:26.617519 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 414/847] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:26 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1743 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:26.633827 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-58d51abf-0eda-4f4f-b569-cf242210689a req-60bd177b-3500-40b4-8e28-1bf64ffd9bd0 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0033415c-3adf-4fe8-bac6-60cf7774143f", "tag": "eacb6063-c62f-4742-8ca6-7f93d25fdaf8"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:46:26.647887 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.objects.instance [None req-aea5c0c8-9a86-49ec-8f2f-2a99fd433d64 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lazy-loading 'services' on Instance uuid 0033415c-3adf-4fe8-bac6-60cf7774143f {{(pid=130159) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:46:26.658947 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-58d51abf-0eda-4f4f-b569-cf242210689a req-60bd177b-3500-40b4-8e28-1bf64ffd9bd0 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:26.660307 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-58d51abf-0eda-4f4f-b569-cf242210689a req-60bd177b-3500-40b4-8e28-1bf64ffd9bd0 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:26.693532 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-58d51abf-0eda-4f4f-b569-cf242210689a req-60bd177b-3500-40b4-8e28-1bf64ffd9bd0 service nova] Creating event network-changed:eacb6063-c62f-4742-8ca6-7f93d25fdaf8 for instance 0033415c-3adf-4fe8-bac6-60cf7774143f on hv-cinder-maste Jul 11 14:46:26.700520 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-58d51abf-0eda-4f4f-b569-cf242210689a req-60bd177b-3500-40b4-8e28-1bf64ffd9bd0 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072464 Jul 11 14:46:26.701576 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 415/848] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:46:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:46:26.802063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-aea5c0c8-9a86-49ec-8f2f-2a99fd433d64 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "DELETE /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/os-volume_attachments/c528a5d6-d787-4620-9e4d-83555a31a506" status: 202 len: 0 microversion: 2.1 time: 0.303750 Jul 11 14:46:26.804642 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 434/849] 10.222.0.72 () {58 vars in 1415 bytes} [Mon Jul 11 14:46:26 2022] DELETE /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/os-volume_attachments/c528a5d6-d787-4620-9e4d-83555a31a506 => generated 0 bytes in 309 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:46:26.817190 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-160cb88a-1fb5-4ec1-9c47-1bfbd0a40b15 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:26.818273 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-160cb88a-1fb5-4ec1-9c47-1bfbd0a40b15 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:26.839244 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-160cb88a-1fb5-4ec1-9c47-1bfbd0a40b15 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:26.841131 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-160cb88a-1fb5-4ec1-9c47-1bfbd0a40b15 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:26.843010 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-160cb88a-1fb5-4ec1-9c47-1bfbd0a40b15 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:26.845814 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-160cb88a-1fb5-4ec1-9c47-1bfbd0a40b15 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:26.942949 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-160cb88a-1fb5-4ec1-9c47-1bfbd0a40b15 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.131117 Jul 11 14:46:26.944941 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 416/850] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:46:26 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/os-volume_attachments => generated 197 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:46:27.654365 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-de6c3dc6-5dd2-4d35-a469-ecf57c2a2fbc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:27.665258 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-de6c3dc6-5dd2-4d35-a469-ecf57c2a2fbc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:27.686065 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-de6c3dc6-5dd2-4d35-a469-ecf57c2a2fbc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:27.686407 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-de6c3dc6-5dd2-4d35-a469-ecf57c2a2fbc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:27.688557 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-de6c3dc6-5dd2-4d35-a469-ecf57c2a2fbc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:27.688557 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-de6c3dc6-5dd2-4d35-a469-ecf57c2a2fbc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:27.900374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-5566cf32-84c5-4f56-8743-720f8f1942a8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:27.901316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-5566cf32-84c5-4f56-8743-720f8f1942a8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:27.911082 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5566cf32-84c5-4f56-8743-720f8f1942a8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:27.911816 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5566cf32-84c5-4f56-8743-720f8f1942a8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:27.912895 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5566cf32-84c5-4f56-8743-720f8f1942a8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:27.913572 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5566cf32-84c5-4f56-8743-720f8f1942a8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:27.955184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-5566cf32-84c5-4f56-8743-720f8f1942a8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/ips" status: 200 len: 147 microversion: 2.1 time: 0.065588 Jul 11 14:46:27.955815 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 417/851] 10.222.0.72 () {58 vars in 1247 bytes} [Mon Jul 11 14:46:27 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/ips => generated 147 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:46:27.965737 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f7233351-3c70-4bf8-a60f-d5877ab22ff5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:27.976169 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f7233351-3c70-4bf8-a60f-d5877ab22ff5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:27.991670 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f7233351-3c70-4bf8-a60f-d5877ab22ff5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:27.993393 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f7233351-3c70-4bf8-a60f-d5877ab22ff5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:27.994830 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f7233351-3c70-4bf8-a60f-d5877ab22ff5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:27.995495 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f7233351-3c70-4bf8-a60f-d5877ab22ff5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:28.081605 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f7233351-3c70-4bf8-a60f-d5877ab22ff5 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.123185 Jul 11 14:46:28.084809 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 418/852] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:46:27 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/os-volume_attachments => generated 197 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:46:28.148471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-de6c3dc6-5dd2-4d35-a469-ecf57c2a2fbc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:28.199618 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-de6c3dc6-5dd2-4d35-a469-ecf57c2a2fbc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:28.215013 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-de6c3dc6-5dd2-4d35-a469-ecf57c2a2fbc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:28.230467 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-de6c3dc6-5dd2-4d35-a469-ecf57c2a2fbc tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1743 microversion: 2.20 time: 0.582490 Jul 11 14:46:28.249470 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 435/853] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:27 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1743 bytes in 605 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:28.577973 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-bac86c35-eb5c-46dd-a215-9b5456230d09 req-96b33069-191e-46ec-bccd-cab27976780b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "43772fe7-b391-4878-bbea-658602d822ad", "name": "network-vif-plugged", "status": "completed", "tag": "35522aa1-427b-4c21-88e8-ce45535459a0"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:46:28.618797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-bac86c35-eb5c-46dd-a215-9b5456230d09 req-96b33069-191e-46ec-bccd-cab27976780b service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:28.619261 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-bac86c35-eb5c-46dd-a215-9b5456230d09 req-96b33069-191e-46ec-bccd-cab27976780b service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:28.661621 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-bac86c35-eb5c-46dd-a215-9b5456230d09 req-96b33069-191e-46ec-bccd-cab27976780b service nova] Creating event network-vif-plugged:35522aa1-427b-4c21-88e8-ce45535459a0 for instance 43772fe7-b391-4878-bbea-658602d822ad on hv-cinder-maste Jul 11 14:46:28.670264 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-bac86c35-eb5c-46dd-a215-9b5456230d09 req-96b33069-191e-46ec-bccd-cab27976780b service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.109511 Jul 11 14:46:28.671528 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 419/854] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:46:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:46:29.102294 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e664d49c-51f7-4389-8cdd-2a57749e5065 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:29.104225 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e664d49c-51f7-4389-8cdd-2a57749e5065 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:29.113507 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e664d49c-51f7-4389-8cdd-2a57749e5065 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:29.114904 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e664d49c-51f7-4389-8cdd-2a57749e5065 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:29.116686 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e664d49c-51f7-4389-8cdd-2a57749e5065 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:29.117502 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e664d49c-51f7-4389-8cdd-2a57749e5065 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:29.152486 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:46:29.178501 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e664d49c-51f7-4389-8cdd-2a57749e5065 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.082349 Jul 11 14:46:29.179936 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 436/855] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:46:29 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/os-volume_attachments => generated 25 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 11 14:46:29.195218 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-dfe4610c-a669-41a5-b211-34d7884ae864 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:29.196131 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-dfe4610c-a669-41a5-b211-34d7884ae864 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:29.211140 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dfe4610c-a669-41a5-b211-34d7884ae864 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:29.212086 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dfe4610c-a669-41a5-b211-34d7884ae864 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:29.213188 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dfe4610c-a669-41a5-b211-34d7884ae864 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:29.213754 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dfe4610c-a669-41a5-b211-34d7884ae864 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:29.288344 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-806b9271-abb9-41c1-9b07-ae82bda1deb8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:29.289532 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-806b9271-abb9-41c1-9b07-ae82bda1deb8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:29.316677 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-806b9271-abb9-41c1-9b07-ae82bda1deb8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:29.322880 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-806b9271-abb9-41c1-9b07-ae82bda1deb8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:29.324316 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-806b9271-abb9-41c1-9b07-ae82bda1deb8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:29.325100 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-806b9271-abb9-41c1-9b07-ae82bda1deb8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:29.413796 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-dfe4610c-a669-41a5-b211-34d7884ae864 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] HTTP exception thrown: Instance 0033415c-3adf-4fe8-bac6-60cf7774143f is not attached to volume c528a5d6-d787-4620-9e4d-83555a31a506 Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-dfe4610c-a669-41a5-b211-34d7884ae864 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Request method failure captured: Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: DELETE /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/os-volume_attachments/c528a5d6-d787-4620-9e4d-83555a31a506 HTTP/1.1 Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDb1iK3wpH396KldybZM_BZTrcGq9AB2iVDkgfsD9xeWPAHSJKXPz42aNMdGnee3L8LHMLJcKKmk_2OAEvE9ppL4P8Ndlctue-vESlUUgk_ZJ88AM1ISoQZ913lQZJOzNj2gEVQpk-E4_MM-qOjrHrANiP1KCv6WRVrIIKcgXPwLEF8 Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: 3e77cfd4544c45468c57510de6034e38 Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-TestMinimumBasicScenario-1092437220 Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:46:29.417700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:46:29.420195 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/3e77cfd4544c45468c57510de6034e38"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/3e77cfd4544c45468c57510de6034e38"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_3e77cfd4544c45468c57510de6034e38"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/3e77cfd4544c45468c57510de6034e38"}]}] Jul 11 14:46:29.420195 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-TestMinimumBasicScenario-1092437220 Jul 11 14:46:29.420195 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: 3e77cfd4544c45468c57510de6034e38 Jul 11 14:46:29.420195 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1092437220 Jul 11 14:46:29.420195 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-TestMinimumBasicScenario-1092437220-project Jul 11 14:46:29.420195 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:46:29.420195 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:46:29.420195 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 82025bd919ce404b96491a19262d4281 Jul 11 14:46:29.420195 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-TestMinimumBasicScenario-1092437220-project Jul 11 14:46:29.420195 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:46:29.420195 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance 0033415c-3adf-4fe8-bac6-60cf7774143f is not attached to volume c528a5d6-d787-4620-9e4d-83555a31a506 Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'server_id': '0033415c-3adf-4fe8-bac6-60cf7774143f', 'id': 'c528a5d6-d787-4620-9e4d-83555a31a506'} Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 281, in get_by_volume_and_instance Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id c528a5d6-d787-4620-9e4d-83555a31a506. Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:46:29.420955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:46:29.422309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:46:29.422309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Jul 11 14:46:29.422309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Jul 11 14:46:29.422309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0033415c-3adf-4fe8-bac6-60cf7774143f is not attached to volume c528a5d6-d787-4620-9e4d-83555a31a506 Jul 11 14:46:29.422309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:46:29.422309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:46:29.422309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:46:29.422309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:46:29.422309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:46:29.422309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:46:29.422309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:46:29.422309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:46:29.422309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0033415c-3adf-4fe8-bac6-60cf7774143f is not attached to volume c528a5d6-d787-4620-9e4d-83555a31a506 Jul 11 14:46:29.422309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:46:29.425159 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-dfe4610c-a669-41a5-b211-34d7884ae864 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Returning 404 to user: Instance 0033415c-3adf-4fe8-bac6-60cf7774143f is not attached to volume c528a5d6-d787-4620-9e4d-83555a31a506 {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:46:29.431619 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-dfe4610c-a669-41a5-b211-34d7884ae864 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "DELETE /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/os-volume_attachments/c528a5d6-d787-4620-9e4d-83555a31a506" status: 404 len: 154 microversion: 2.1 time: 0.242210 Jul 11 14:46:29.432374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 420/856] 10.222.0.72 () {58 vars in 1415 bytes} [Mon Jul 11 14:46:29 2022] DELETE /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f/os-volume_attachments/c528a5d6-d787-4620-9e4d-83555a31a506 => generated 154 bytes in 244 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 11 14:46:29.719230 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-806b9271-abb9-41c1-9b07-ae82bda1deb8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:29.741200 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-806b9271-abb9-41c1-9b07-ae82bda1deb8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:29.746679 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-806b9271-abb9-41c1-9b07-ae82bda1deb8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:29.773878 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-806b9271-abb9-41c1-9b07-ae82bda1deb8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1738 microversion: 2.20 time: 0.483101 Jul 11 14:46:29.792397 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-ca4d7291-bc8d-4ce1-b749-8519fdc5e32b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2eca43d0-4021-4c3c-8c78-f6d7ef72c914"}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:46:29.795541 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 437/857] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:29 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1738 bytes in 519 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:29.796257 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-ca4d7291-bc8d-4ce1-b749-8519fdc5e32b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:29.816277 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ca4d7291-bc8d-4ce1-b749-8519fdc5e32b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:29.817418 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ca4d7291-bc8d-4ce1-b749-8519fdc5e32b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:29.819052 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ca4d7291-bc8d-4ce1-b749-8519fdc5e32b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:29.819962 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ca4d7291-bc8d-4ce1-b749-8519fdc5e32b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:30.304703 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-ca4d7291-bc8d-4ce1-b749-8519fdc5e32b tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "POST /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.526268 Jul 11 14:46:30.308045 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 421/858] 10.222.0.72 () {60 vars in 1322 bytes} [Mon Jul 11 14:46:29 2022] POST /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments => generated 194 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:46:30.760661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d93443b7-ffe3-4b10-bd82-6a817ead18b4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:30.761869 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d93443b7-ffe3-4b10-bd82-6a817ead18b4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:30.774127 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d93443b7-ffe3-4b10-bd82-6a817ead18b4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:30.775224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d93443b7-ffe3-4b10-bd82-6a817ead18b4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:30.776656 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d93443b7-ffe3-4b10-bd82-6a817ead18b4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:30.777867 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d93443b7-ffe3-4b10-bd82-6a817ead18b4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:30.827945 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d93443b7-ffe3-4b10-bd82-6a817ead18b4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Going to try to terminate instance {{(pid=130159) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:46:30.836075 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-15f0d7aa-25e6-4400-bed1-cbfbb07ed083 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:30.840278 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-15f0d7aa-25e6-4400-bed1-cbfbb07ed083 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:30.866652 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-15f0d7aa-25e6-4400-bed1-cbfbb07ed083 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:30.867453 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-15f0d7aa-25e6-4400-bed1-cbfbb07ed083 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:30.880471 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-15f0d7aa-25e6-4400-bed1-cbfbb07ed083 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:30.881404 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-15f0d7aa-25e6-4400-bed1-cbfbb07ed083 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:31.002483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-d93443b7-ffe3-4b10-bd82-6a817ead18b4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "DELETE /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 204 len: 0 microversion: 2.1 time: 0.255627 Jul 11 14:46:31.003161 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 438/859] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:46:30 2022] DELETE /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 11 14:46:31.013452 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-37f61f37-beba-4b1b-a651-54f81204a7c4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:31.014792 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-37f61f37-beba-4b1b-a651-54f81204a7c4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:31.027294 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-37f61f37-beba-4b1b-a651-54f81204a7c4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:31.028126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-37f61f37-beba-4b1b-a651-54f81204a7c4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:31.029214 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-37f61f37-beba-4b1b-a651-54f81204a7c4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:31.030929 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-37f61f37-beba-4b1b-a651-54f81204a7c4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:31.154086 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-15f0d7aa-25e6-4400-bed1-cbfbb07ed083 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:31.163674 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-15f0d7aa-25e6-4400-bed1-cbfbb07ed083 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:31.165375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-15f0d7aa-25e6-4400-bed1-cbfbb07ed083 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:31.166616 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-15f0d7aa-25e6-4400-bed1-cbfbb07ed083 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1738 microversion: 2.20 time: 0.339823 Jul 11 14:46:31.167544 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 422/860] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:30 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1738 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:31.259050 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-37f61f37-beba-4b1b-a651-54f81204a7c4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:31.285609 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-37f61f37-beba-4b1b-a651-54f81204a7c4 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1552 microversion: 2.1 time: 0.270339 Jul 11 14:46:31.288725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 439/861] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:46:31 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1552 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:46:32.197974 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-5c81be12-38ec-4fe4-9a9a-9b914e7fe087 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:32.199677 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-5c81be12-38ec-4fe4-9a9a-9b914e7fe087 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:32.208973 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5c81be12-38ec-4fe4-9a9a-9b914e7fe087 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:32.209793 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5c81be12-38ec-4fe4-9a9a-9b914e7fe087 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:32.210645 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5c81be12-38ec-4fe4-9a9a-9b914e7fe087 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:32.211088 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5c81be12-38ec-4fe4-9a9a-9b914e7fe087 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:32.322947 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-0d824db6-97ca-4bd3-a371-2966aaa0d163 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:32.324980 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-0d824db6-97ca-4bd3-a371-2966aaa0d163 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:32.339455 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0d824db6-97ca-4bd3-a371-2966aaa0d163 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:32.340284 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0d824db6-97ca-4bd3-a371-2966aaa0d163 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:32.341633 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0d824db6-97ca-4bd3-a371-2966aaa0d163 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:32.342303 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0d824db6-97ca-4bd3-a371-2966aaa0d163 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:32.428761 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5c81be12-38ec-4fe4-9a9a-9b914e7fe087 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:32.442201 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5c81be12-38ec-4fe4-9a9a-9b914e7fe087 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:32.444358 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5c81be12-38ec-4fe4-9a9a-9b914e7fe087 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:32.445631 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-5c81be12-38ec-4fe4-9a9a-9b914e7fe087 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1738 microversion: 2.20 time: 0.251721 Jul 11 14:46:32.446843 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 423/862] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:32 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1738 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:32.542015 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0d824db6-97ca-4bd3-a371-2966aaa0d163 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:32.552344 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-0d824db6-97ca-4bd3-a371-2966aaa0d163 tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1552 microversion: 2.1 time: 0.234260 Jul 11 14:46:32.553188 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 440/863] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:46:32 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1552 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:46:33.465642 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-629765b5-b8f8-478d-9fa0-9bed2bcf67ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:33.466207 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-629765b5-b8f8-478d-9fa0-9bed2bcf67ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:33.474962 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-629765b5-b8f8-478d-9fa0-9bed2bcf67ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:33.475941 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-629765b5-b8f8-478d-9fa0-9bed2bcf67ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:33.476750 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-629765b5-b8f8-478d-9fa0-9bed2bcf67ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:33.477496 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-629765b5-b8f8-478d-9fa0-9bed2bcf67ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:33.574504 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-865da3ea-0a46-412f-9c4d-647b0465eaea tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:33.576891 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-865da3ea-0a46-412f-9c4d-647b0465eaea tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:33.591210 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-865da3ea-0a46-412f-9c4d-647b0465eaea tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:33.592705 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-865da3ea-0a46-412f-9c4d-647b0465eaea tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:33.594767 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-865da3ea-0a46-412f-9c4d-647b0465eaea tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:33.595959 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-865da3ea-0a46-412f-9c4d-647b0465eaea tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:33.665572 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-629765b5-b8f8-478d-9fa0-9bed2bcf67ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:33.684377 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-629765b5-b8f8-478d-9fa0-9bed2bcf67ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:33.685928 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-629765b5-b8f8-478d-9fa0-9bed2bcf67ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:33.687187 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-629765b5-b8f8-478d-9fa0-9bed2bcf67ae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1738 microversion: 2.20 time: 0.225913 Jul 11 14:46:33.688082 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 424/864] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:33 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1738 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:46:34.018503 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-865da3ea-0a46-412f-9c4d-647b0465eaea tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '82025bd919ce404b96491a19262d4281', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3e77cfd4544c45468c57510de6034e38', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:34.042850 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-865da3ea-0a46-412f-9c4d-647b0465eaea tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 200 len: 1552 microversion: 2.1 time: 0.475113 Jul 11 14:46:34.058488 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 441/865] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:46:33 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 1552 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:46:34.195640 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-1c469ecd-a804-46ce-8ca3-4dfe6d2ba7af req-a5ff95d2-bcd7-44fc-a63b-1e6b891bf87e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0033415c-3adf-4fe8-bac6-60cf7774143f", "name": "network-vif-deleted", "tag": "eacb6063-c62f-4742-8ca6-7f93d25fdaf8"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:46:34.215076 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-1c469ecd-a804-46ce-8ca3-4dfe6d2ba7af req-a5ff95d2-bcd7-44fc-a63b-1e6b891bf87e service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:34.216102 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-1c469ecd-a804-46ce-8ca3-4dfe6d2ba7af req-a5ff95d2-bcd7-44fc-a63b-1e6b891bf87e service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:34.243014 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-1c469ecd-a804-46ce-8ca3-4dfe6d2ba7af req-a5ff95d2-bcd7-44fc-a63b-1e6b891bf87e service nova] Creating event network-vif-deleted:eacb6063-c62f-4742-8ca6-7f93d25fdaf8 for instance 0033415c-3adf-4fe8-bac6-60cf7774143f on hv-cinder-maste Jul 11 14:46:34.249731 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-1c469ecd-a804-46ce-8ca3-4dfe6d2ba7af req-a5ff95d2-bcd7-44fc-a63b-1e6b891bf87e service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059102 Jul 11 14:46:34.251111 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 425/866] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:46:34 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:46:34.709991 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8acec6a6-f8cc-4d5b-a145-69e492583ca8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:34.711357 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8acec6a6-f8cc-4d5b-a145-69e492583ca8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:34.722464 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8acec6a6-f8cc-4d5b-a145-69e492583ca8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:34.723618 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8acec6a6-f8cc-4d5b-a145-69e492583ca8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:34.725295 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8acec6a6-f8cc-4d5b-a145-69e492583ca8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:34.726473 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8acec6a6-f8cc-4d5b-a145-69e492583ca8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:35.046357 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8acec6a6-f8cc-4d5b-a145-69e492583ca8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:35.070198 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8acec6a6-f8cc-4d5b-a145-69e492583ca8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:35.088910 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8acec6a6-f8cc-4d5b-a145-69e492583ca8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:35.089399 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-1acf733a-ce22-4135-8ac2-d51b1c615d5d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:35.091052 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-1acf733a-ce22-4135-8ac2-d51b1c615d5d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] [instance: 0033415c-3adf-4fe8-bac6-60cf7774143f] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:35.093131 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8acec6a6-f8cc-4d5b-a145-69e492583ca8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1738 microversion: 2.20 time: 0.388535 Jul 11 14:46:35.095064 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 442/867] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:34 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1738 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:35.105076 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1acf733a-ce22-4135-8ac2-d51b1c615d5d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:35.105723 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1acf733a-ce22-4135-8ac2-d51b1c615d5d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:35.107118 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1acf733a-ce22-4135-8ac2-d51b1c615d5d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:35.107680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1acf733a-ce22-4135-8ac2-d51b1c615d5d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:35.136865 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-1acf733a-ce22-4135-8ac2-d51b1c615d5d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] HTTP exception thrown: Instance 0033415c-3adf-4fe8-bac6-60cf7774143f could not be found. Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-1acf733a-ce22-4135-8ac2-d51b1c615d5d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Request method failure captured: Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f HTTP/1.1 Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDb1iK3wpH396KldybZM_BZTrcGq9AB2iVDkgfsD9xeWPAHSJKXPz42aNMdGnee3L8LHMLJcKKmk_2OAEvE9ppL4P8Ndlctue-vESlUUgk_ZJ88AM1ISoQZ913lQZJOzNj2gEVQpk-E4_MM-qOjrHrANiP1KCv6WRVrIIKcgXPwLEF8 Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: 3e77cfd4544c45468c57510de6034e38 Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-TestMinimumBasicScenario-1092437220 Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:46:35.140164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:46:35.142988 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/3e77cfd4544c45468c57510de6034e38"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/3e77cfd4544c45468c57510de6034e38"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_3e77cfd4544c45468c57510de6034e38"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/3e77cfd4544c45468c57510de6034e38"}]}] Jul 11 14:46:35.142988 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-TestMinimumBasicScenario-1092437220 Jul 11 14:46:35.142988 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: 3e77cfd4544c45468c57510de6034e38 Jul 11 14:46:35.142988 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-TestMinimumBasicScenario-1092437220 Jul 11 14:46:35.142988 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-TestMinimumBasicScenario-1092437220-project Jul 11 14:46:35.142988 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:46:35.142988 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:46:35.142988 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 82025bd919ce404b96491a19262d4281 Jul 11 14:46:35.142988 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-TestMinimumBasicScenario-1092437220-project Jul 11 14:46:35.142988 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:46:35.142988 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance 0033415c-3adf-4fe8-bac6-60cf7774143f could not be found. Jul 11 14:46:35.142988 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': '0033415c-3adf-4fe8-bac6-60cf7774143f'} Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0033415c-3adf-4fe8-bac6-60cf7774143f could not be found. Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:46:35.144961 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:46:35.148299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:46:35.148299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:46:35.148299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:46:35.148299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:46:35.148299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:46:35.148299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:46:35.148299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:46:35.148299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0033415c-3adf-4fe8-bac6-60cf7774143f could not be found. Jul 11 14:46:35.148299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:46:35.148299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:46:35.148299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:46:35.148299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:46:35.148299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:46:35.148299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:46:35.148299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:46:35.148299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:46:35.148299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0033415c-3adf-4fe8-bac6-60cf7774143f could not be found. Jul 11 14:46:35.148299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:46:35.152352 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-1acf733a-ce22-4135-8ac2-d51b1c615d5d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Returning 404 to user: Instance 0033415c-3adf-4fe8-bac6-60cf7774143f could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:46:35.152352 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-1acf733a-ce22-4135-8ac2-d51b1c615d5d tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f" status: 404 len: 111 microversion: 2.1 time: 0.064126 Jul 11 14:46:35.157774 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2dae7534-187d-42a7-9536-580a9fd81cac tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] Calling method '.version_select at 0x7f592af37a60>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:35.158552 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 426/868] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:46:35 2022] GET /compute/v2.1/servers/0033415c-3adf-4fe8-bac6-60cf7774143f => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:46:35.202325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2dae7534-187d-42a7-9536-580a9fd81cac tempest-TestMinimumBasicScenario-1092437220 tempest-TestMinimumBasicScenario-1092437220-project] 10.222.0.72 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-749000618" status: 202 len: 0 microversion: 2.1 time: 0.048668 Jul 11 14:46:35.224488 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 443/869] 10.222.0.72 () {58 vars in 1268 bytes} [Mon Jul 11 14:46:35 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-749000618 => generated 0 bytes in 73 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:46:36.118973 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e4973d9f-81f1-483b-9116-b60af77b0a96 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:36.121712 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-e4973d9f-81f1-483b-9116-b60af77b0a96 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:36.149714 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e4973d9f-81f1-483b-9116-b60af77b0a96 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:36.151135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e4973d9f-81f1-483b-9116-b60af77b0a96 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:36.153474 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e4973d9f-81f1-483b-9116-b60af77b0a96 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:36.154697 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e4973d9f-81f1-483b-9116-b60af77b0a96 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:36.478663 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e4973d9f-81f1-483b-9116-b60af77b0a96 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:36.495169 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e4973d9f-81f1-483b-9116-b60af77b0a96 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:36.499080 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e4973d9f-81f1-483b-9116-b60af77b0a96 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:36.512439 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e4973d9f-81f1-483b-9116-b60af77b0a96 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1738 microversion: 2.20 time: 0.401654 Jul 11 14:46:36.518535 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 427/870] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:36 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1738 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:37.548358 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8a681181-a17a-4cf0-b9e8-1be3eb1dbd63 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:37.549879 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8a681181-a17a-4cf0-b9e8-1be3eb1dbd63 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:37.561462 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8a681181-a17a-4cf0-b9e8-1be3eb1dbd63 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:37.562215 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8a681181-a17a-4cf0-b9e8-1be3eb1dbd63 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:37.563123 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8a681181-a17a-4cf0-b9e8-1be3eb1dbd63 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:37.564043 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8a681181-a17a-4cf0-b9e8-1be3eb1dbd63 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:37.995118 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8a681181-a17a-4cf0-b9e8-1be3eb1dbd63 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:38.015939 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8a681181-a17a-4cf0-b9e8-1be3eb1dbd63 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:38.026699 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-8a681181-a17a-4cf0-b9e8-1be3eb1dbd63 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:38.028659 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8a681181-a17a-4cf0-b9e8-1be3eb1dbd63 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1738 microversion: 2.20 time: 0.484950 Jul 11 14:46:38.043119 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 444/871] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:37 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1738 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:38.357668 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-cd6b6bce-fc6f-4a5b-bf57-f820e09943c7 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:38.360725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-cd6b6bce-fc6f-4a5b-bf57-f820e09943c7 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:38.387564 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cd6b6bce-fc6f-4a5b-bf57-f820e09943c7 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:38.388402 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cd6b6bce-fc6f-4a5b-bf57-f820e09943c7 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:38.389917 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cd6b6bce-fc6f-4a5b-bf57-f820e09943c7 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:38.390747 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cd6b6bce-fc6f-4a5b-bf57-f820e09943c7 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:38.484277 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-cd6b6bce-fc6f-4a5b-bf57-f820e09943c7 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.131775 Jul 11 14:46:38.494692 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 428/872] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:46:38 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments => generated 197 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:46:38.501305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a84a9fbd-7a1f-426b-87ed-d1f86299da6c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:38.502474 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a84a9fbd-7a1f-426b-87ed-d1f86299da6c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:38.512578 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a84a9fbd-7a1f-426b-87ed-d1f86299da6c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:38.513432 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a84a9fbd-7a1f-426b-87ed-d1f86299da6c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:38.514468 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a84a9fbd-7a1f-426b-87ed-d1f86299da6c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:38.515146 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a84a9fbd-7a1f-426b-87ed-d1f86299da6c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:38.623488 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a84a9fbd-7a1f-426b-87ed-d1f86299da6c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments/2eca43d0-4021-4c3c-8c78-f6d7ef72c914" status: 200 len: 194 microversion: 2.1 time: 0.128102 Jul 11 14:46:38.628690 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 445/873] 10.222.0.72 () {58 vars in 1412 bytes} [Mon Jul 11 14:46:38 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments/2eca43d0-4021-4c3c-8c78-f6d7ef72c914 => generated 194 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:46:39.077821 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-1077123d-ae3c-424b-bfe9-3444c51c635d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:39.084046 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-1077123d-ae3c-424b-bfe9-3444c51c635d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:39.103012 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1077123d-ae3c-424b-bfe9-3444c51c635d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:39.104440 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1077123d-ae3c-424b-bfe9-3444c51c635d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:39.105757 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1077123d-ae3c-424b-bfe9-3444c51c635d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:39.106252 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-1077123d-ae3c-424b-bfe9-3444c51c635d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:39.563191 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-1077123d-ae3c-424b-bfe9-3444c51c635d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:39.598165 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-1077123d-ae3c-424b-bfe9-3444c51c635d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:39.604734 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-1077123d-ae3c-424b-bfe9-3444c51c635d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:39.608316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-1077123d-ae3c-424b-bfe9-3444c51c635d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1738 microversion: 2.20 time: 0.538380 Jul 11 14:46:39.610202 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 429/874] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:39 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1738 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:40.518254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9bda9754-c0bf-4a2e-b471-b121f1e7cffc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "5a6389ed-c77d-4246-88c3-ad78d3479941"}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:46:40.523368 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9bda9754-c0bf-4a2e-b471-b121f1e7cffc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:40.551202 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9bda9754-c0bf-4a2e-b471-b121f1e7cffc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:40.554455 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9bda9754-c0bf-4a2e-b471-b121f1e7cffc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:40.562358 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9bda9754-c0bf-4a2e-b471-b121f1e7cffc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:40.563408 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9bda9754-c0bf-4a2e-b471-b121f1e7cffc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:40.644839 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-73f12132-f337-4c48-8362-230cfea1333e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:40.645981 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-73f12132-f337-4c48-8362-230cfea1333e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:40.691535 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=50 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:46:40.695509 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-73f12132-f337-4c48-8362-230cfea1333e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:40.705227 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-73f12132-f337-4c48-8362-230cfea1333e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:40.707398 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-73f12132-f337-4c48-8362-230cfea1333e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:40.708450 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-73f12132-f337-4c48-8362-230cfea1333e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:41.182651 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-73f12132-f337-4c48-8362-230cfea1333e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:41.215206 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-73f12132-f337-4c48-8362-230cfea1333e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:41.220000 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-73f12132-f337-4c48-8362-230cfea1333e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:41.224564 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-73f12132-f337-4c48-8362-230cfea1333e tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1738 microversion: 2.20 time: 0.587049 Jul 11 14:46:41.227663 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 430/875] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:40 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1738 bytes in 592 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:41.387215 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9bda9754-c0bf-4a2e-b471-b121f1e7cffc tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "POST /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.877891 Jul 11 14:46:41.388452 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 446/876] 10.222.0.72 () {60 vars in 1322 bytes} [Mon Jul 11 14:46:40 2022] POST /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments => generated 194 bytes in 881 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:46:42.257829 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a8d26914-f8da-455e-ae64-4e009ab715a0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:42.258701 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a8d26914-f8da-455e-ae64-4e009ab715a0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:42.282128 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a8d26914-f8da-455e-ae64-4e009ab715a0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:42.282638 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a8d26914-f8da-455e-ae64-4e009ab715a0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:42.283478 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a8d26914-f8da-455e-ae64-4e009ab715a0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:42.283986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a8d26914-f8da-455e-ae64-4e009ab715a0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:42.659508 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a8d26914-f8da-455e-ae64-4e009ab715a0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:42.700699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a8d26914-f8da-455e-ae64-4e009ab715a0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:42.706952 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a8d26914-f8da-455e-ae64-4e009ab715a0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:42.711446 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a8d26914-f8da-455e-ae64-4e009ab715a0 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1738 microversion: 2.20 time: 0.458137 Jul 11 14:46:42.716895 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 431/877] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:42 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1738 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:43.760147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-54ea2020-d4dc-41a8-87ef-0edc6bc6393c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:43.772696 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-54ea2020-d4dc-41a8-87ef-0edc6bc6393c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:43.791288 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-54ea2020-d4dc-41a8-87ef-0edc6bc6393c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:43.792987 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-54ea2020-d4dc-41a8-87ef-0edc6bc6393c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:43.795140 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-54ea2020-d4dc-41a8-87ef-0edc6bc6393c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:43.796565 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-54ea2020-d4dc-41a8-87ef-0edc6bc6393c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:44.405992 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-54ea2020-d4dc-41a8-87ef-0edc6bc6393c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:44.465134 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-54ea2020-d4dc-41a8-87ef-0edc6bc6393c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:44.467876 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-54ea2020-d4dc-41a8-87ef-0edc6bc6393c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:44.469489 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-54ea2020-d4dc-41a8-87ef-0edc6bc6393c tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1738 microversion: 2.20 time: 0.720598 Jul 11 14:46:44.481157 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 447/878] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:43 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1738 bytes in 740 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:45.494127 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-aa69b901-6597-41db-a890-e49eafc83c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:45.495275 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-aa69b901-6597-41db-a890-e49eafc83c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:45.509167 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-aa69b901-6597-41db-a890-e49eafc83c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:45.510332 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-aa69b901-6597-41db-a890-e49eafc83c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:45.512045 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-aa69b901-6597-41db-a890-e49eafc83c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:45.512811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-aa69b901-6597-41db-a890-e49eafc83c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:45.849818 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-aa69b901-6597-41db-a890-e49eafc83c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:45.856309 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:46:45.888306 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-aa69b901-6597-41db-a890-e49eafc83c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:45.892306 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-aa69b901-6597-41db-a890-e49eafc83c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:45.895786 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-aa69b901-6597-41db-a890-e49eafc83c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1738 microversion: 2.20 time: 0.409818 Jul 11 14:46:45.910471 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 432/879] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:45 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1738 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:46.931043 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-15d1fe1b-d37c-496f-9096-7022ce714205 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:46.934953 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-15d1fe1b-d37c-496f-9096-7022ce714205 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:46.950730 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-15d1fe1b-d37c-496f-9096-7022ce714205 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:46.951986 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-15d1fe1b-d37c-496f-9096-7022ce714205 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:46.954258 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-15d1fe1b-d37c-496f-9096-7022ce714205 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:46.955614 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-15d1fe1b-d37c-496f-9096-7022ce714205 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:47.222848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-15d1fe1b-d37c-496f-9096-7022ce714205 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:47.251975 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-15d1fe1b-d37c-496f-9096-7022ce714205 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:47.254869 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-15d1fe1b-d37c-496f-9096-7022ce714205 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:47.256122 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-15d1fe1b-d37c-496f-9096-7022ce714205 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1720 microversion: 2.20 time: 0.333630 Jul 11 14:46:47.256821 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 448/880] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:46 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1720 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:48.282317 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-5f0dca73-af10-45f5-95b4-f1b8426f6d4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:48.283459 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-5f0dca73-af10-45f5-95b4-f1b8426f6d4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:48.316115 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5f0dca73-af10-45f5-95b4-f1b8426f6d4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:48.316945 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5f0dca73-af10-45f5-95b4-f1b8426f6d4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:48.318485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5f0dca73-af10-45f5-95b4-f1b8426f6d4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:48.323848 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5f0dca73-af10-45f5-95b4-f1b8426f6d4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:48.680701 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-2851ab96-b5c5-4468-be7a-440a1430bd8c req-ea0b6278-7881-4507-8aa1-6d2ef61b7fa5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "43772fe7-b391-4878-bbea-658602d822ad", "name": "network-vif-unplugged", "status": "completed", "tag": "35522aa1-427b-4c21-88e8-ce45535459a0"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:46:48.750161 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-2851ab96-b5c5-4468-be7a-440a1430bd8c req-ea0b6278-7881-4507-8aa1-6d2ef61b7fa5 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:48.760869 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-2851ab96-b5c5-4468-be7a-440a1430bd8c req-ea0b6278-7881-4507-8aa1-6d2ef61b7fa5 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:48.815164 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-2851ab96-b5c5-4468-be7a-440a1430bd8c req-ea0b6278-7881-4507-8aa1-6d2ef61b7fa5 service nova] Creating event network-vif-unplugged:35522aa1-427b-4c21-88e8-ce45535459a0 for instance 43772fe7-b391-4878-bbea-658602d822ad on hv-cinder-maste Jul 11 14:46:48.829322 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-2851ab96-b5c5-4468-be7a-440a1430bd8c req-ea0b6278-7881-4507-8aa1-6d2ef61b7fa5 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.178019 Jul 11 14:46:48.831348 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 449/881] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:46:48 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:46:48.876843 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5f0dca73-af10-45f5-95b4-f1b8426f6d4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:48.896081 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5f0dca73-af10-45f5-95b4-f1b8426f6d4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:48.897893 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5f0dca73-af10-45f5-95b4-f1b8426f6d4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:48.899415 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-5f0dca73-af10-45f5-95b4-f1b8426f6d4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1720 microversion: 2.20 time: 0.624738 Jul 11 14:46:48.899996 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 433/882] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:48 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1720 bytes in 627 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:46:49.937484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-77614e2b-e80b-4dfe-bac6-a97905ccdede tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:49.939303 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-77614e2b-e80b-4dfe-bac6-a97905ccdede tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:49.954783 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-77614e2b-e80b-4dfe-bac6-a97905ccdede tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:49.956617 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-77614e2b-e80b-4dfe-bac6-a97905ccdede tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:49.958706 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-77614e2b-e80b-4dfe-bac6-a97905ccdede tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:49.959903 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-77614e2b-e80b-4dfe-bac6-a97905ccdede tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:50.192122 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-77614e2b-e80b-4dfe-bac6-a97905ccdede tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:50.206362 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-77614e2b-e80b-4dfe-bac6-a97905ccdede tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:50.209881 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-77614e2b-e80b-4dfe-bac6-a97905ccdede tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:50.212541 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-77614e2b-e80b-4dfe-bac6-a97905ccdede tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1663 microversion: 2.20 time: 0.280668 Jul 11 14:46:50.219824 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 450/883] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:49 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1663 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:46:50.243734 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7fcbc90b-9f62-438b-a298-63a067d2a8fa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9eaa1c8a-63b5-4239-a5fd-d69b05018c36"}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:46:50.245586 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-7fcbc90b-9f62-438b-a298-63a067d2a8fa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:50.257713 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7fcbc90b-9f62-438b-a298-63a067d2a8fa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:50.258484 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7fcbc90b-9f62-438b-a298-63a067d2a8fa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:50.259801 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7fcbc90b-9f62-438b-a298-63a067d2a8fa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:50.260517 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7fcbc90b-9f62-438b-a298-63a067d2a8fa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:50.773733 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-7fcbc90b-9f62-438b-a298-63a067d2a8fa tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "POST /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.537570 Jul 11 14:46:50.774403 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 434/884] 10.222.0.72 () {62 vars in 1363 bytes} [Mon Jul 11 14:46:50 2022] POST /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad/os-volume_attachments => generated 188 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Jul 11 14:46:50.782660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f772d8e0-157a-4406-8851-e886063c3565 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:50.789837 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f772d8e0-157a-4406-8851-e886063c3565 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:50.800449 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f772d8e0-157a-4406-8851-e886063c3565 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:50.801116 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f772d8e0-157a-4406-8851-e886063c3565 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:50.802830 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f772d8e0-157a-4406-8851-e886063c3565 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:50.803583 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f772d8e0-157a-4406-8851-e886063c3565 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:50.855273 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:46:50.864182 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f772d8e0-157a-4406-8851-e886063c3565 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.088874 Jul 11 14:46:50.865524 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 451/885] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:46:50 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments => generated 371 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:46:50.996149 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-b09c40a8-06ac-42a6-8695-23e7aabb0f40 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:50.996601 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-b09c40a8-06ac-42a6-8695-23e7aabb0f40 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:51.010803 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b09c40a8-06ac-42a6-8695-23e7aabb0f40 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:51.010803 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b09c40a8-06ac-42a6-8695-23e7aabb0f40 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:51.012103 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b09c40a8-06ac-42a6-8695-23e7aabb0f40 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:51.013191 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b09c40a8-06ac-42a6-8695-23e7aabb0f40 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:51.020616 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9e4ae786-8eb5-4737-98bf-3afd137559be tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:51.021469 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9e4ae786-8eb5-4737-98bf-3afd137559be tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:51.031570 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9e4ae786-8eb5-4737-98bf-3afd137559be tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:51.032483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9e4ae786-8eb5-4737-98bf-3afd137559be tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:51.033930 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9e4ae786-8eb5-4737-98bf-3afd137559be tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:51.034755 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9e4ae786-8eb5-4737-98bf-3afd137559be tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:51.097729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9e4ae786-8eb5-4737-98bf-3afd137559be tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments/2eca43d0-4021-4c3c-8c78-f6d7ef72c914" status: 200 len: 194 microversion: 2.1 time: 0.082701 Jul 11 14:46:51.099624 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 452/886] 10.222.0.72 () {58 vars in 1412 bytes} [Mon Jul 11 14:46:51 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments/2eca43d0-4021-4c3c-8c78-f6d7ef72c914 => generated 194 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:46:51.122098 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-6f434660-4de5-459f-9001-6e0993845367 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:51.123540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-6f434660-4de5-459f-9001-6e0993845367 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:51.136933 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6f434660-4de5-459f-9001-6e0993845367 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:51.138559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6f434660-4de5-459f-9001-6e0993845367 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:51.141252 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6f434660-4de5-459f-9001-6e0993845367 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:51.142481 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6f434660-4de5-459f-9001-6e0993845367 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:51.318035 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.objects.instance [None req-6f434660-4de5-459f-9001-6e0993845367 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lazy-loading 'services' on Instance uuid cb814dfe-6200-40ed-832f-df4face7a16e {{(pid=130159) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:46:51.462217 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-6f434660-4de5-459f-9001-6e0993845367 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "DELETE /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments/2eca43d0-4021-4c3c-8c78-f6d7ef72c914" status: 202 len: 0 microversion: 2.1 time: 0.351266 Jul 11 14:46:51.463773 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 453/887] 10.222.0.72 () {58 vars in 1415 bytes} [Mon Jul 11 14:46:51 2022] DELETE /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments/2eca43d0-4021-4c3c-8c78-f6d7ef72c914 => generated 0 bytes in 354 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:46:51.567828 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-b09c40a8-06ac-42a6-8695-23e7aabb0f40 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "DELETE /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad/os-volume_attachments/9eaa1c8a-63b5-4239-a5fd-d69b05018c36" status: 202 len: 0 microversion: 2.20 time: 0.575399 Jul 11 14:46:51.570219 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 435/888] 10.222.0.72 () {60 vars in 1456 bytes} [Mon Jul 11 14:46:50 2022] DELETE /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad/os-volume_attachments/9eaa1c8a-63b5-4239-a5fd-d69b05018c36 => generated 0 bytes in 579 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Jul 11 14:46:51.717866 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-97bbc6d0-a24d-4bd8-8aab-5234f37f00d6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:46:51.720196 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-97bbc6d0-a24d-4bd8-8aab-5234f37f00d6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:51.729480 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-97bbc6d0-a24d-4bd8-8aab-5234f37f00d6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:51.730219 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-97bbc6d0-a24d-4bd8-8aab-5234f37f00d6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:51.731350 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-97bbc6d0-a24d-4bd8-8aab-5234f37f00d6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:51.731941 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-97bbc6d0-a24d-4bd8-8aab-5234f37f00d6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:51.857104 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-97bbc6d0-a24d-4bd8-8aab-5234f37f00d6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "POST /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad/action" status: 202 len: 0 microversion: 2.20 time: 0.148051 Jul 11 14:46:51.858471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 454/889] 10.222.0.72 () {62 vars in 1318 bytes} [Mon Jul 11 14:46:51 2022] POST /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad/action => generated 0 bytes in 151 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Jul 11 14:46:51.871629 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2b399a85-8c16-496e-a3f0-88ac2daeacb6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:51.872771 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2b399a85-8c16-496e-a3f0-88ac2daeacb6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:51.884551 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2b399a85-8c16-496e-a3f0-88ac2daeacb6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:51.885404 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2b399a85-8c16-496e-a3f0-88ac2daeacb6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:51.886923 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2b399a85-8c16-496e-a3f0-88ac2daeacb6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:51.887700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2b399a85-8c16-496e-a3f0-88ac2daeacb6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:52.098783 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2b399a85-8c16-496e-a3f0-88ac2daeacb6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:52.109780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2b399a85-8c16-496e-a3f0-88ac2daeacb6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:52.111349 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2b399a85-8c16-496e-a3f0-88ac2daeacb6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:52.114081 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2b399a85-8c16-496e-a3f0-88ac2daeacb6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1671 microversion: 2.20 time: 0.249425 Jul 11 14:46:52.116688 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 436/890] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:51 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1671 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:52.725440 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-28f025c2-5ec5-4ef7-8a0f-449b2856b940 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:52.726059 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-28f025c2-5ec5-4ef7-8a0f-449b2856b940 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:52.738646 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-28f025c2-5ec5-4ef7-8a0f-449b2856b940 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:52.739353 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-28f025c2-5ec5-4ef7-8a0f-449b2856b940 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:52.740776 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-28f025c2-5ec5-4ef7-8a0f-449b2856b940 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:52.741436 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-28f025c2-5ec5-4ef7-8a0f-449b2856b940 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:52.801576 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-28f025c2-5ec5-4ef7-8a0f-449b2856b940 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments/5a6389ed-c77d-4246-88c3-ad78d3479941" status: 200 len: 194 microversion: 2.1 time: 0.082912 Jul 11 14:46:52.802655 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 455/891] 10.222.0.72 () {58 vars in 1412 bytes} [Mon Jul 11 14:46:52 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments/5a6389ed-c77d-4246-88c3-ad78d3479941 => generated 194 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:46:52.819963 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c975a9e8-01b0-4ea6-a511-9f6e0c7151d4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:52.820874 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c975a9e8-01b0-4ea6-a511-9f6e0c7151d4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:52.833317 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c975a9e8-01b0-4ea6-a511-9f6e0c7151d4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:52.833933 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c975a9e8-01b0-4ea6-a511-9f6e0c7151d4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:52.834959 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c975a9e8-01b0-4ea6-a511-9f6e0c7151d4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:52.835517 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c975a9e8-01b0-4ea6-a511-9f6e0c7151d4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:52.976971 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.objects.instance [None req-c975a9e8-01b0-4ea6-a511-9f6e0c7151d4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lazy-loading 'services' on Instance uuid cb814dfe-6200-40ed-832f-df4face7a16e {{(pid=130160) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:46:53.063234 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c975a9e8-01b0-4ea6-a511-9f6e0c7151d4 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "DELETE /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments/5a6389ed-c77d-4246-88c3-ad78d3479941" status: 202 len: 0 microversion: 2.1 time: 0.251311 Jul 11 14:46:53.064034 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 437/892] 10.222.0.72 () {58 vars in 1415 bytes} [Mon Jul 11 14:46:52 2022] DELETE /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments/5a6389ed-c77d-4246-88c3-ad78d3479941 => generated 0 bytes in 254 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:46:53.144067 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-963f756c-5dbe-4b98-b97a-5b64bac09a23 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:53.144783 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-963f756c-5dbe-4b98-b97a-5b64bac09a23 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:53.152317 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-963f756c-5dbe-4b98-b97a-5b64bac09a23 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:53.152763 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-963f756c-5dbe-4b98-b97a-5b64bac09a23 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:53.153706 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-963f756c-5dbe-4b98-b97a-5b64bac09a23 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:53.153993 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-963f756c-5dbe-4b98-b97a-5b64bac09a23 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:53.417760 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-963f756c-5dbe-4b98-b97a-5b64bac09a23 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:53.433698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-963f756c-5dbe-4b98-b97a-5b64bac09a23 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:53.436579 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-963f756c-5dbe-4b98-b97a-5b64bac09a23 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:53.438778 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-963f756c-5dbe-4b98-b97a-5b64bac09a23 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1729 microversion: 2.20 time: 0.298976 Jul 11 14:46:53.439821 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 456/893] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:53 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1729 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:54.352302 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-628cf260-16d2-4ece-8078-2435d92b5312 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:54.353482 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-628cf260-16d2-4ece-8078-2435d92b5312 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:54.361797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-628cf260-16d2-4ece-8078-2435d92b5312 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:54.362883 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-628cf260-16d2-4ece-8078-2435d92b5312 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:54.364351 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-628cf260-16d2-4ece-8078-2435d92b5312 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:54.365138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-628cf260-16d2-4ece-8078-2435d92b5312 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:54.407117 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-628cf260-16d2-4ece-8078-2435d92b5312 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.072856 Jul 11 14:46:54.407721 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 438/894] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:46:54 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments => generated 25 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 11 14:46:54.470807 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-49712370-f318-454f-a605-eb9771879f3b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:54.471623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-49712370-f318-454f-a605-eb9771879f3b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:54.482797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-49712370-f318-454f-a605-eb9771879f3b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:54.483256 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-49712370-f318-454f-a605-eb9771879f3b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:54.484046 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-49712370-f318-454f-a605-eb9771879f3b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:54.484488 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-49712370-f318-454f-a605-eb9771879f3b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:54.573859 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-57e3dd56-2112-49b3-b45f-0a50ff7b1a6f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:54.575778 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-57e3dd56-2112-49b3-b45f-0a50ff7b1a6f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:54.590042 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-57e3dd56-2112-49b3-b45f-0a50ff7b1a6f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:54.591119 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-57e3dd56-2112-49b3-b45f-0a50ff7b1a6f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:54.592019 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-57e3dd56-2112-49b3-b45f-0a50ff7b1a6f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:54.592576 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-57e3dd56-2112-49b3-b45f-0a50ff7b1a6f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:54.643848 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-57e3dd56-2112-49b3-b45f-0a50ff7b1a6f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.077252 Jul 11 14:46:54.644823 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 439/895] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:46:54 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e/os-volume_attachments => generated 25 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Jul 11 14:46:54.709856 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-49712370-f318-454f-a605-eb9771879f3b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:54.726658 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-49712370-f318-454f-a605-eb9771879f3b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:54.729240 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-49712370-f318-454f-a605-eb9771879f3b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:54.732489 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-49712370-f318-454f-a605-eb9771879f3b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1729 microversion: 2.20 time: 0.266365 Jul 11 14:46:54.733067 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 457/896] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:54 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1729 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:54.750743 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2530b625-abdf-44c6-92c8-214e9447b112 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:54.751441 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2530b625-abdf-44c6-92c8-214e9447b112 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:54.758980 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2530b625-abdf-44c6-92c8-214e9447b112 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:54.759579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2530b625-abdf-44c6-92c8-214e9447b112 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:54.760831 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2530b625-abdf-44c6-92c8-214e9447b112 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:54.761003 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2530b625-abdf-44c6-92c8-214e9447b112 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:54.789451 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2530b625-abdf-44c6-92c8-214e9447b112 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Going to try to terminate instance {{(pid=130160) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:46:54.900098 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2530b625-abdf-44c6-92c8-214e9447b112 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "DELETE /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 204 len: 0 microversion: 2.1 time: 0.153629 Jul 11 14:46:54.901331 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 440/897] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:46:54 2022] DELETE /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 11 14:46:54.912980 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1972c824-7cb5-4902-a04f-d447f57f066f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:54.914230 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1972c824-7cb5-4902-a04f-d447f57f066f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:54.919755 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=58,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:46:54.934211 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1972c824-7cb5-4902-a04f-d447f57f066f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:54.935470 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1972c824-7cb5-4902-a04f-d447f57f066f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:54.937344 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1972c824-7cb5-4902-a04f-d447f57f066f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:54.938558 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1972c824-7cb5-4902-a04f-d447f57f066f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:55.129105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1972c824-7cb5-4902-a04f-d447f57f066f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:55.137440 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1972c824-7cb5-4902-a04f-d447f57f066f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1675 microversion: 2.1 time: 0.231373 Jul 11 14:46:55.138033 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 458/898] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:46:54 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1675 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:46:55.660112 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a5d2a234-1dda-4e39-bff4-38ebc5ca55e5 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:55.660983 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a5d2a234-1dda-4e39-bff4-38ebc5ca55e5 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:55.669050 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a5d2a234-1dda-4e39-bff4-38ebc5ca55e5 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:55.669526 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a5d2a234-1dda-4e39-bff4-38ebc5ca55e5 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:55.670291 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a5d2a234-1dda-4e39-bff4-38ebc5ca55e5 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:55.670772 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a5d2a234-1dda-4e39-bff4-38ebc5ca55e5 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:55.701345 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a5d2a234-1dda-4e39-bff4-38ebc5ca55e5 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b/ips" status: 200 len: 149 microversion: 2.1 time: 0.045959 Jul 11 14:46:55.701943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 441/899] 10.222.0.72 () {58 vars in 1247 bytes} [Mon Jul 11 14:46:55 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b/ips => generated 149 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:46:55.757588 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-6033cd27-0c2a-4133-83fd-c1146bc40287 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:55.759819 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-6033cd27-0c2a-4133-83fd-c1146bc40287 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:55.772646 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6033cd27-0c2a-4133-83fd-c1146bc40287 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:55.773450 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6033cd27-0c2a-4133-83fd-c1146bc40287 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:55.775185 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6033cd27-0c2a-4133-83fd-c1146bc40287 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:55.776154 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6033cd27-0c2a-4133-83fd-c1146bc40287 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:55.974312 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-6033cd27-0c2a-4133-83fd-c1146bc40287 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:55.988503 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-6033cd27-0c2a-4133-83fd-c1146bc40287 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:55.991916 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-6033cd27-0c2a-4133-83fd-c1146bc40287 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:55.994874 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-6033cd27-0c2a-4133-83fd-c1146bc40287 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1729 microversion: 2.20 time: 0.245145 Jul 11 14:46:55.996050 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 459/900] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:55 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1729 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:56.158721 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-6aba2e24-64eb-4aed-a2d6-657784b8cdcf tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:56.159451 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-6aba2e24-64eb-4aed-a2d6-657784b8cdcf tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:56.167107 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6aba2e24-64eb-4aed-a2d6-657784b8cdcf tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:56.167706 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6aba2e24-64eb-4aed-a2d6-657784b8cdcf tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:56.168584 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6aba2e24-64eb-4aed-a2d6-657784b8cdcf tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:56.169138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6aba2e24-64eb-4aed-a2d6-657784b8cdcf tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:56.175078 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=84,nova_cell1:INSERT=7,nova_cell1:UPDATE=9 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:46:56.330505 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6aba2e24-64eb-4aed-a2d6-657784b8cdcf tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:56.341131 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-6aba2e24-64eb-4aed-a2d6-657784b8cdcf tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1675 microversion: 2.1 time: 0.187263 Jul 11 14:46:56.341822 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 442/901] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:46:56 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1675 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:46:57.031350 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-5c353c41-d9b6-4fb8-86e5-6533036adbae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:57.033564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-5c353c41-d9b6-4fb8-86e5-6533036adbae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:57.059389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5c353c41-d9b6-4fb8-86e5-6533036adbae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:57.060556 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5c353c41-d9b6-4fb8-86e5-6533036adbae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:57.062320 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5c353c41-d9b6-4fb8-86e5-6533036adbae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:57.063456 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-5c353c41-d9b6-4fb8-86e5-6533036adbae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:57.250571 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8de0f471-05f8-4d06-ab99-7aaa32e3cb2b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "8a7d6bfe-a5be-4a10-a855-683b92af94a5"}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:46:57.254649 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-8de0f471-05f8-4d06-ab99-7aaa32e3cb2b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:57.291202 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8de0f471-05f8-4d06-ab99-7aaa32e3cb2b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:57.293905 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8de0f471-05f8-4d06-ab99-7aaa32e3cb2b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:57.296001 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8de0f471-05f8-4d06-ab99-7aaa32e3cb2b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:57.297163 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8de0f471-05f8-4d06-ab99-7aaa32e3cb2b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:57.352550 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-5c353c41-d9b6-4fb8-86e5-6533036adbae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:57.371879 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-5c353c41-d9b6-4fb8-86e5-6533036adbae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:57.376748 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-5c353c41-d9b6-4fb8-86e5-6533036adbae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:57.379615 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-5c353c41-d9b6-4fb8-86e5-6533036adbae tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1729 microversion: 2.20 time: 0.357149 Jul 11 14:46:57.382870 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 460/902] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:57 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1729 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:46:57.392469 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-56b99c3a-8760-40e1-9de3-f20ee3ded48a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:57.396520 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-56b99c3a-8760-40e1-9de3-f20ee3ded48a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:57.409901 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-56b99c3a-8760-40e1-9de3-f20ee3ded48a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:57.410783 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-56b99c3a-8760-40e1-9de3-f20ee3ded48a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:57.412007 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-56b99c3a-8760-40e1-9de3-f20ee3ded48a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:57.412962 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-56b99c3a-8760-40e1-9de3-f20ee3ded48a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:57.723544 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-56b99c3a-8760-40e1-9de3-f20ee3ded48a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:57.732001 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-56b99c3a-8760-40e1-9de3-f20ee3ded48a tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1675 microversion: 2.1 time: 0.347728 Jul 11 14:46:57.732598 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 461/903] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:46:57 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1675 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:46:58.413915 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f5a42753-9a67-4ef0-8266-511d55cc43b7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:58.416783 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f5a42753-9a67-4ef0-8266-511d55cc43b7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:58.428416 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f5a42753-9a67-4ef0-8266-511d55cc43b7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:58.429173 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f5a42753-9a67-4ef0-8266-511d55cc43b7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:58.430578 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f5a42753-9a67-4ef0-8266-511d55cc43b7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:58.431456 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f5a42753-9a67-4ef0-8266-511d55cc43b7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:58.647126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f5a42753-9a67-4ef0-8266-511d55cc43b7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:58.667086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f5a42753-9a67-4ef0-8266-511d55cc43b7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:58.673256 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f5a42753-9a67-4ef0-8266-511d55cc43b7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:58.676641 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f5a42753-9a67-4ef0-8266-511d55cc43b7 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1729 microversion: 2.20 time: 0.270726 Jul 11 14:46:58.693453 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 462/904] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:58 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1729 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:46:58.758185 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-0eab8d2b-fc7e-4deb-8cdf-0cdac5f74e00 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:58.759968 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-0eab8d2b-fc7e-4deb-8cdf-0cdac5f74e00 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:58.774540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0eab8d2b-fc7e-4deb-8cdf-0cdac5f74e00 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:58.775440 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0eab8d2b-fc7e-4deb-8cdf-0cdac5f74e00 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:58.777876 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0eab8d2b-fc7e-4deb-8cdf-0cdac5f74e00 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:58.779040 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0eab8d2b-fc7e-4deb-8cdf-0cdac5f74e00 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:58.991731 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0eab8d2b-fc7e-4deb-8cdf-0cdac5f74e00 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:59.002115 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-0eab8d2b-fc7e-4deb-8cdf-0cdac5f74e00 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1675 microversion: 2.1 time: 0.254826 Jul 11 14:46:59.003730 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 463/905] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:46:58 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1675 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:46:59.709621 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e5b7a6af-0ce9-4db3-8fbf-ae07ee23d47d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:46:59.711577 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e5b7a6af-0ce9-4db3-8fbf-ae07ee23d47d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:46:59.722765 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e5b7a6af-0ce9-4db3-8fbf-ae07ee23d47d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:59.723536 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e5b7a6af-0ce9-4db3-8fbf-ae07ee23d47d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:59.724947 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e5b7a6af-0ce9-4db3-8fbf-ae07ee23d47d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:46:59.725707 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e5b7a6af-0ce9-4db3-8fbf-ae07ee23d47d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:46:59.976693 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e5b7a6af-0ce9-4db3-8fbf-ae07ee23d47d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:59.992193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e5b7a6af-0ce9-4db3-8fbf-ae07ee23d47d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:59.995526 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e5b7a6af-0ce9-4db3-8fbf-ae07ee23d47d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:46:59.998283 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e5b7a6af-0ce9-4db3-8fbf-ae07ee23d47d tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1729 microversion: 2.20 time: 0.295999 Jul 11 14:46:59.999576 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 464/906] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:46:59 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1729 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:47:00.020976 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-4a0ff538-02a2-4d51-bb1f-d2e0eedc4d11 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:00.021609 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-4a0ff538-02a2-4d51-bb1f-d2e0eedc4d11 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:00.028922 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4a0ff538-02a2-4d51-bb1f-d2e0eedc4d11 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:00.029377 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4a0ff538-02a2-4d51-bb1f-d2e0eedc4d11 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:00.030174 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4a0ff538-02a2-4d51-bb1f-d2e0eedc4d11 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:00.030657 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4a0ff538-02a2-4d51-bb1f-d2e0eedc4d11 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:00.249743 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-4a0ff538-02a2-4d51-bb1f-d2e0eedc4d11 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:00.270990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-4a0ff538-02a2-4d51-bb1f-d2e0eedc4d11 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1675 microversion: 2.1 time: 0.253705 Jul 11 14:47:00.272087 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 465/907] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:00 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1675 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:47:01.029495 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e030ff75-d0f4-431a-922f-6db5ae670431 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:01.030766 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e030ff75-d0f4-431a-922f-6db5ae670431 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:01.039874 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e030ff75-d0f4-431a-922f-6db5ae670431 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:01.040994 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e030ff75-d0f4-431a-922f-6db5ae670431 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:01.042143 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e030ff75-d0f4-431a-922f-6db5ae670431 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:01.043005 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e030ff75-d0f4-431a-922f-6db5ae670431 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:01.326880 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e030ff75-d0f4-431a-922f-6db5ae670431 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:01.336138 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:47:01.348203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e030ff75-d0f4-431a-922f-6db5ae670431 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:01.351905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e030ff75-d0f4-431a-922f-6db5ae670431 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:01.355023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e030ff75-d0f4-431a-922f-6db5ae670431 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1729 microversion: 2.20 time: 0.331778 Jul 11 14:47:01.357334 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 466/908] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:47:01 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1729 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:47:01.366808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-84925efc-e877-44b5-8fa9-00ffe762ee46 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:01.368363 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-84925efc-e877-44b5-8fa9-00ffe762ee46 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:01.381506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84925efc-e877-44b5-8fa9-00ffe762ee46 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:01.382810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84925efc-e877-44b5-8fa9-00ffe762ee46 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:01.384874 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84925efc-e877-44b5-8fa9-00ffe762ee46 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:01.385969 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84925efc-e877-44b5-8fa9-00ffe762ee46 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:01.685851 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-84925efc-e877-44b5-8fa9-00ffe762ee46 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:01.702850 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-84925efc-e877-44b5-8fa9-00ffe762ee46 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1675 microversion: 2.1 time: 0.342858 Jul 11 14:47:01.717751 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 467/909] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:01 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1675 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:47:02.384468 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-53dc1f05-6a7f-4c0d-9d15-f32e5309988f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:02.385530 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-53dc1f05-6a7f-4c0d-9d15-f32e5309988f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:02.398080 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-53dc1f05-6a7f-4c0d-9d15-f32e5309988f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:02.398981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-53dc1f05-6a7f-4c0d-9d15-f32e5309988f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:02.400311 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-53dc1f05-6a7f-4c0d-9d15-f32e5309988f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:02.401106 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-53dc1f05-6a7f-4c0d-9d15-f32e5309988f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:02.696328 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-53dc1f05-6a7f-4c0d-9d15-f32e5309988f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:02.716005 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-53dc1f05-6a7f-4c0d-9d15-f32e5309988f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:02.721966 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-53dc1f05-6a7f-4c0d-9d15-f32e5309988f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:02.725030 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-53dc1f05-6a7f-4c0d-9d15-f32e5309988f tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1729 microversion: 2.20 time: 0.344417 Jul 11 14:47:02.725824 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 468/910] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:47:02 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1729 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:47:02.740774 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2cd213a9-5d9e-4ef6-96b3-12d3cec23e54 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:02.742534 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2cd213a9-5d9e-4ef6-96b3-12d3cec23e54 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:02.751816 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2cd213a9-5d9e-4ef6-96b3-12d3cec23e54 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:02.752483 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2cd213a9-5d9e-4ef6-96b3-12d3cec23e54 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:02.753452 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2cd213a9-5d9e-4ef6-96b3-12d3cec23e54 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:02.753951 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2cd213a9-5d9e-4ef6-96b3-12d3cec23e54 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:02.909626 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-2cd213a9-5d9e-4ef6-96b3-12d3cec23e54 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:02.923401 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2cd213a9-5d9e-4ef6-96b3-12d3cec23e54 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1675 microversion: 2.1 time: 0.191837 Jul 11 14:47:02.924464 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 469/911] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:02 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1675 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:47:03.759267 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c4ff0b0e-3787-4740-b651-ee94693cf096 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:03.760559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c4ff0b0e-3787-4740-b651-ee94693cf096 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:03.773429 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c4ff0b0e-3787-4740-b651-ee94693cf096 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:03.774235 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c4ff0b0e-3787-4740-b651-ee94693cf096 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:03.775732 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c4ff0b0e-3787-4740-b651-ee94693cf096 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:03.776505 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c4ff0b0e-3787-4740-b651-ee94693cf096 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:03.994034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c4ff0b0e-3787-4740-b651-ee94693cf096 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:04.003306 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c4ff0b0e-3787-4740-b651-ee94693cf096 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:04.005066 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c4ff0b0e-3787-4740-b651-ee94693cf096 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:04.006278 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c4ff0b0e-3787-4740-b651-ee94693cf096 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1729 microversion: 2.20 time: 0.255707 Jul 11 14:47:04.006867 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 470/912] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:47:03 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1729 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:47:04.011945 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-ffbdce4c-faed-4d30-8f41-ef5d17d77ac8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:04.012827 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-ffbdce4c-faed-4d30-8f41-ef5d17d77ac8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:04.019753 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ffbdce4c-faed-4d30-8f41-ef5d17d77ac8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:04.020513 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ffbdce4c-faed-4d30-8f41-ef5d17d77ac8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:04.021495 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ffbdce4c-faed-4d30-8f41-ef5d17d77ac8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:04.022114 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ffbdce4c-faed-4d30-8f41-ef5d17d77ac8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:04.228139 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-ffbdce4c-faed-4d30-8f41-ef5d17d77ac8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:04.236413 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-ffbdce4c-faed-4d30-8f41-ef5d17d77ac8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1675 microversion: 2.1 time: 0.228126 Jul 11 14:47:04.237927 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 471/913] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:04 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1675 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:47:05.029555 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-883ff71a-9294-427f-bd61-29899a315c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:05.030975 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-883ff71a-9294-427f-bd61-29899a315c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:05.042579 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-883ff71a-9294-427f-bd61-29899a315c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:05.043372 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-883ff71a-9294-427f-bd61-29899a315c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:05.044790 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-883ff71a-9294-427f-bd61-29899a315c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:05.045599 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-883ff71a-9294-427f-bd61-29899a315c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:05.220145 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-883ff71a-9294-427f-bd61-29899a315c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:05.228992 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-883ff71a-9294-427f-bd61-29899a315c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:05.230603 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-883ff71a-9294-427f-bd61-29899a315c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:05.231842 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-883ff71a-9294-427f-bd61-29899a315c38 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1729 microversion: 2.20 time: 0.209656 Jul 11 14:47:05.232425 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 472/914] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:47:05 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1729 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:47:05.262366 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f27cf602-ee6f-4169-8749-aa8cbfd0715f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:05.263163 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f27cf602-ee6f-4169-8749-aa8cbfd0715f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:05.270261 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f27cf602-ee6f-4169-8749-aa8cbfd0715f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:05.270758 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f27cf602-ee6f-4169-8749-aa8cbfd0715f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:05.271566 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f27cf602-ee6f-4169-8749-aa8cbfd0715f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:05.272023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f27cf602-ee6f-4169-8749-aa8cbfd0715f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:05.425802 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f27cf602-ee6f-4169-8749-aa8cbfd0715f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:05.435099 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f27cf602-ee6f-4169-8749-aa8cbfd0715f tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1675 microversion: 2.1 time: 0.177010 Jul 11 14:47:05.435698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 473/915] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:05 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1675 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:47:06.251756 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9367b209-3420-46cc-b558-3989ac424262 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:06.253491 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9367b209-3420-46cc-b558-3989ac424262 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:06.260938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9367b209-3420-46cc-b558-3989ac424262 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:06.261436 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9367b209-3420-46cc-b558-3989ac424262 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:06.262761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9367b209-3420-46cc-b558-3989ac424262 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:06.263457 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9367b209-3420-46cc-b558-3989ac424262 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:06.409393 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9367b209-3420-46cc-b558-3989ac424262 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:06.425385 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9367b209-3420-46cc-b558-3989ac424262 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:06.428629 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9367b209-3420-46cc-b558-3989ac424262 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:06.430989 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9367b209-3420-46cc-b558-3989ac424262 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1729 microversion: 2.20 time: 0.183540 Jul 11 14:47:06.432292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 474/916] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:47:06 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1729 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:47:06.467427 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-fe1cea82-db8e-426b-8dbf-c3638ce27cfd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:06.468996 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-fe1cea82-db8e-426b-8dbf-c3638ce27cfd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:06.480216 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fe1cea82-db8e-426b-8dbf-c3638ce27cfd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:06.483001 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fe1cea82-db8e-426b-8dbf-c3638ce27cfd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:06.483897 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fe1cea82-db8e-426b-8dbf-c3638ce27cfd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:06.484354 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fe1cea82-db8e-426b-8dbf-c3638ce27cfd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:06.615901 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8de0f471-05f8-4d06-ab99-7aaa32e3cb2b tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "POST /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 9.376757 Jul 11 14:47:06.619161 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 443/917] 10.222.0.72 () {60 vars in 1322 bytes} [Mon Jul 11 14:46:57 2022] POST /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b/os-volume_attachments => generated 194 bytes in 9382 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:47:06.794096 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-fe1cea82-db8e-426b-8dbf-c3638ce27cfd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:06.813144 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-fe1cea82-db8e-426b-8dbf-c3638ce27cfd tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1675 microversion: 2.1 time: 0.354542 Jul 11 14:47:06.813144 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 475/918] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:06 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1675 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:47:07.460841 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-96e1c513-7e3b-49cb-872b-9d4991f6acc1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:07.461820 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-96e1c513-7e3b-49cb-872b-9d4991f6acc1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:07.464619 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=32,nova_api:UPDATE=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:47:07.475523 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-96e1c513-7e3b-49cb-872b-9d4991f6acc1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:07.476273 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-96e1c513-7e3b-49cb-872b-9d4991f6acc1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:07.477306 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-96e1c513-7e3b-49cb-872b-9d4991f6acc1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:07.477942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-96e1c513-7e3b-49cb-872b-9d4991f6acc1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:07.655822 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-96e1c513-7e3b-49cb-872b-9d4991f6acc1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:07.684556 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-2e04954e-058d-4e32-97c8-dff8b3671669 req-91fc4e8a-b3b4-4576-9eec-c7c10e77b793 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb814dfe-6200-40ed-832f-df4face7a16e", "name": "network-vif-deleted", "tag": "acddbdf4-6edc-4eaf-8287-a34936c1b912"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:47:07.698292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-2e04954e-058d-4e32-97c8-dff8b3671669 req-91fc4e8a-b3b4-4576-9eec-c7c10e77b793 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:07.699527 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-2e04954e-058d-4e32-97c8-dff8b3671669 req-91fc4e8a-b3b4-4576-9eec-c7c10e77b793 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:07.710668 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-96e1c513-7e3b-49cb-872b-9d4991f6acc1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:07.714748 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-96e1c513-7e3b-49cb-872b-9d4991f6acc1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:07.721746 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-2e04954e-058d-4e32-97c8-dff8b3671669 req-91fc4e8a-b3b4-4576-9eec-c7c10e77b793 service nova] Creating event network-vif-deleted:acddbdf4-6edc-4eaf-8287-a34936c1b912 for instance cb814dfe-6200-40ed-832f-df4face7a16e on hv-cinder-maste Jul 11 14:47:07.729310 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-2e04954e-058d-4e32-97c8-dff8b3671669 req-91fc4e8a-b3b4-4576-9eec-c7c10e77b793 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057465 Jul 11 14:47:07.729987 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-96e1c513-7e3b-49cb-872b-9d4991f6acc1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1729 microversion: 2.20 time: 0.262238 Jul 11 14:47:07.731064 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 444/919] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:47:07 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1729 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:47:07.736657 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 476/920] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:47:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:47:07.842784 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-966a1cbe-5d36-462d-9b05-c4147d857249 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:07.844660 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-966a1cbe-5d36-462d-9b05-c4147d857249 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:07.857503 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-966a1cbe-5d36-462d-9b05-c4147d857249 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:07.858968 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-966a1cbe-5d36-462d-9b05-c4147d857249 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:07.860848 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-966a1cbe-5d36-462d-9b05-c4147d857249 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:07.861882 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-966a1cbe-5d36-462d-9b05-c4147d857249 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:07.930095 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-966a1cbe-5d36-462d-9b05-c4147d857249 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:07.931033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-966a1cbe-5d36-462d-9b05-c4147d857249 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:07.932182 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.objects.instance [None req-966a1cbe-5d36-462d-9b05-c4147d857249 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lazy-loading 'fault' on Instance uuid cb814dfe-6200-40ed-832f-df4face7a16e {{(pid=130160) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:47:08.000041 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-966a1cbe-5d36-462d-9b05-c4147d857249 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '467992b53c4749858fd8b0ccd2db2393', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c0e363b1584848c1aecbe186cf91eec5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:08.011321 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-966a1cbe-5d36-462d-9b05-c4147d857249 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 200 len: 1328 microversion: 2.1 time: 0.176331 Jul 11 14:47:08.012107 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 445/921] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:07 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 1328 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:47:08.755399 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-14b2723f-614e-476e-bb70-db268ac3e953 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:08.756787 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-14b2723f-614e-476e-bb70-db268ac3e953 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:08.769971 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-14b2723f-614e-476e-bb70-db268ac3e953 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:08.770907 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-14b2723f-614e-476e-bb70-db268ac3e953 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:08.772352 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-14b2723f-614e-476e-bb70-db268ac3e953 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:08.773151 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-14b2723f-614e-476e-bb70-db268ac3e953 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:09.028222 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-14b2723f-614e-476e-bb70-db268ac3e953 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:09.044107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-14b2723f-614e-476e-bb70-db268ac3e953 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:09.045779 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-4a4bc913-887b-414b-a83a-4aeb8618028d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:09.046861 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-4a4bc913-887b-414b-a83a-4aeb8618028d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:09.047453 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-14b2723f-614e-476e-bb70-db268ac3e953 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:09.050236 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-14b2723f-614e-476e-bb70-db268ac3e953 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1729 microversion: 2.20 time: 0.302987 Jul 11 14:47:09.058563 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 477/922] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:47:08 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1729 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:47:09.058854 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4a4bc913-887b-414b-a83a-4aeb8618028d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:09.058854 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4a4bc913-887b-414b-a83a-4aeb8618028d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:09.058854 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4a4bc913-887b-414b-a83a-4aeb8618028d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:09.059517 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4a4bc913-887b-414b-a83a-4aeb8618028d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:09.084104 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-4a4bc913-887b-414b-a83a-4aeb8618028d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] HTTP exception thrown: Instance cb814dfe-6200-40ed-832f-df4face7a16e could not be found. Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-4a4bc913-887b-414b-a83a-4aeb8618028d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Request method failure captured: Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e HTTP/1.1 Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDZgOjz1xpcRYZZu-Kh_5N3i0wpyKor5piLH_ZAuxS7Y7F4dt1mCndOM5aBnPiEJue8x_XFxSfgCnilWaGVpEbgHnMB6NZWPYBeKDQCtfPPAwXFcEuCxxWlEzENjbPi2j267u3jA32MzeWpRtu42O7GlF56dv-e0bwbLODZDNeFgobk Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: c0e363b1584848c1aecbe186cf91eec5 Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-AttachVolumeTestJSON-588294681 Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:47:09.086357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:47:09.089580 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/c0e363b1584848c1aecbe186cf91eec5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/c0e363b1584848c1aecbe186cf91eec5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_c0e363b1584848c1aecbe186cf91eec5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/c0e363b1584848c1aecbe186cf91eec5"}]}] Jul 11 14:47:09.089580 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-AttachVolumeTestJSON-588294681 Jul 11 14:47:09.089580 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: c0e363b1584848c1aecbe186cf91eec5 Jul 11 14:47:09.089580 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-AttachVolumeTestJSON-588294681 Jul 11 14:47:09.089580 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-AttachVolumeTestJSON-588294681-project Jul 11 14:47:09.089580 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:47:09.089580 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:47:09.089580 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 467992b53c4749858fd8b0ccd2db2393 Jul 11 14:47:09.089580 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-AttachVolumeTestJSON-588294681-project Jul 11 14:47:09.089580 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:47:09.089580 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance cb814dfe-6200-40ed-832f-df4face7a16e could not be found. Jul 11 14:47:09.089580 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': 'cb814dfe-6200-40ed-832f-df4face7a16e'} Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cb814dfe-6200-40ed-832f-df4face7a16e could not be found. Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:09.091067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:09.093053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:09.093053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:47:09.093053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:47:09.093053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:47:09.093053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:47:09.093053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:47:09.093053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:47:09.093053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cb814dfe-6200-40ed-832f-df4face7a16e could not be found. Jul 11 14:47:09.093053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:09.093053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:09.093053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:09.093053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:09.093053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:09.093053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:09.093053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:47:09.093053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:47:09.093053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cb814dfe-6200-40ed-832f-df4face7a16e could not be found. Jul 11 14:47:09.093053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:09.094240 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-4a4bc913-887b-414b-a83a-4aeb8618028d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Returning 404 to user: Instance cb814dfe-6200-40ed-832f-df4face7a16e could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:09.094240 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-4a4bc913-887b-414b-a83a-4aeb8618028d tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 404 len: 111 microversion: 2.1 time: 0.060468 Jul 11 14:47:09.094240 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 446/923] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:09 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 111 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 11 14:47:09.101341 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-7dae04c1-63d3-47ad-9f54-b3665d0fc5b7 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '.version_select at 0x7f5930223ee0>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:09.129845 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-7dae04c1-63d3-47ad-9f54-b3665d0fc5b7 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1923133962" status: 202 len: 0 microversion: 2.1 time: 0.032574 Jul 11 14:47:09.136338 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 478/924] 10.222.0.72 () {58 vars in 1220 bytes} [Mon Jul 11 14:47:09 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1923133962 => generated 0 bytes in 40 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 11 14:47:10.074114 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a97d401d-3cf2-4c25-b7f1-0e32ed10e6e1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:10.074977 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a97d401d-3cf2-4c25-b7f1-0e32ed10e6e1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:10.092247 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a97d401d-3cf2-4c25-b7f1-0e32ed10e6e1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:10.093308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a97d401d-3cf2-4c25-b7f1-0e32ed10e6e1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:10.096618 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a97d401d-3cf2-4c25-b7f1-0e32ed10e6e1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:10.098298 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a97d401d-3cf2-4c25-b7f1-0e32ed10e6e1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:10.287280 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a97d401d-3cf2-4c25-b7f1-0e32ed10e6e1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:10.296181 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a97d401d-3cf2-4c25-b7f1-0e32ed10e6e1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:10.297736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a97d401d-3cf2-4c25-b7f1-0e32ed10e6e1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:10.299578 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a97d401d-3cf2-4c25-b7f1-0e32ed10e6e1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1729 microversion: 2.20 time: 0.228993 Jul 11 14:47:10.300394 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 447/925] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:47:10 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1729 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:47:10.510453 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-2ec9025f-6184-48d8-a224-7708fcdf2d7c req-3c2cbec1-eaba-4c51-b276-2b7ad3cff361 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "43772fe7-b391-4878-bbea-658602d822ad", "name": "network-vif-plugged", "status": "completed", "tag": "35522aa1-427b-4c21-88e8-ce45535459a0"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:47:10.522732 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-2ec9025f-6184-48d8-a224-7708fcdf2d7c req-3c2cbec1-eaba-4c51-b276-2b7ad3cff361 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:10.523245 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-2ec9025f-6184-48d8-a224-7708fcdf2d7c req-3c2cbec1-eaba-4c51-b276-2b7ad3cff361 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:10.546365 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-2ec9025f-6184-48d8-a224-7708fcdf2d7c req-3c2cbec1-eaba-4c51-b276-2b7ad3cff361 service nova] Creating event network-vif-plugged:35522aa1-427b-4c21-88e8-ce45535459a0 for instance 43772fe7-b391-4878-bbea-658602d822ad on hv-cinder-maste Jul 11 14:47:10.557943 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-2ec9025f-6184-48d8-a224-7708fcdf2d7c req-3c2cbec1-eaba-4c51-b276-2b7ad3cff361 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053009 Jul 11 14:47:10.558580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 479/926] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:47:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:47:11.319779 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-fa83a20d-853f-4c98-af8a-eeaf50c80f4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:11.320785 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-fa83a20d-853f-4c98-af8a-eeaf50c80f4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:11.328771 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fa83a20d-853f-4c98-af8a-eeaf50c80f4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:11.329296 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fa83a20d-853f-4c98-af8a-eeaf50c80f4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:11.330213 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fa83a20d-853f-4c98-af8a-eeaf50c80f4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:11.330749 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fa83a20d-853f-4c98-af8a-eeaf50c80f4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:11.501467 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-fa83a20d-853f-4c98-af8a-eeaf50c80f4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:11.510137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-fa83a20d-853f-4c98-af8a-eeaf50c80f4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:11.511722 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-fa83a20d-853f-4c98-af8a-eeaf50c80f4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:11.512936 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-fa83a20d-853f-4c98-af8a-eeaf50c80f4b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1729 microversion: 2.20 time: 0.198629 Jul 11 14:47:11.513521 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 448/927] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:47:11 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1729 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:47:12.533224 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-7ea020bb-051a-430e-8bb8-ec4111786f7b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:12.533947 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-7ea020bb-051a-430e-8bb8-ec4111786f7b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:12.541087 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7ea020bb-051a-430e-8bb8-ec4111786f7b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:12.541607 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7ea020bb-051a-430e-8bb8-ec4111786f7b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:12.542477 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7ea020bb-051a-430e-8bb8-ec4111786f7b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:12.542972 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7ea020bb-051a-430e-8bb8-ec4111786f7b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:12.702272 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7ea020bb-051a-430e-8bb8-ec4111786f7b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:12.712233 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7ea020bb-051a-430e-8bb8-ec4111786f7b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:12.713792 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7ea020bb-051a-430e-8bb8-ec4111786f7b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:12.715067 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-7ea020bb-051a-430e-8bb8-ec4111786f7b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1729 microversion: 2.20 time: 0.186986 Jul 11 14:47:12.715658 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 480/928] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:47:12 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1729 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:47:13.734105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-419cd0f9-f74c-4ba3-befd-bf7ecca8e1e8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:13.735544 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-419cd0f9-f74c-4ba3-befd-bf7ecca8e1e8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:13.744648 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-419cd0f9-f74c-4ba3-befd-bf7ecca8e1e8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:13.745171 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-419cd0f9-f74c-4ba3-befd-bf7ecca8e1e8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:13.745967 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-419cd0f9-f74c-4ba3-befd-bf7ecca8e1e8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:13.746412 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-419cd0f9-f74c-4ba3-befd-bf7ecca8e1e8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:13.933694 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-419cd0f9-f74c-4ba3-befd-bf7ecca8e1e8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:13.944298 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-419cd0f9-f74c-4ba3-befd-bf7ecca8e1e8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:13.946136 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-419cd0f9-f74c-4ba3-befd-bf7ecca8e1e8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:13.947694 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-419cd0f9-f74c-4ba3-befd-bf7ecca8e1e8 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1729 microversion: 2.20 time: 0.217687 Jul 11 14:47:13.948434 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 449/929] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:47:13 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1729 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:47:14.604372 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-cd2e2013-ed47-4e00-94ad-9cf128ce77c8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:14.605103 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-cd2e2013-ed47-4e00-94ad-9cf128ce77c8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:14.612475 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cd2e2013-ed47-4e00-94ad-9cf128ce77c8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:14.612930 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cd2e2013-ed47-4e00-94ad-9cf128ce77c8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:14.613712 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cd2e2013-ed47-4e00-94ad-9cf128ce77c8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:14.614181 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-cd2e2013-ed47-4e00-94ad-9cf128ce77c8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:14.633982 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-cd2e2013-ed47-4e00-94ad-9cf128ce77c8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] HTTP exception thrown: Instance cb814dfe-6200-40ed-832f-df4face7a16e could not be found. Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-cd2e2013-ed47-4e00-94ad-9cf128ce77c8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Request method failure captured: Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: DELETE /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e HTTP/1.1 Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZgOjz1xpcRYZZu-Kh_5N3i0wpyKor5piLH_ZAuxS7Y7F4dt1mCndOM5aBnPiEJue8x_XFxSfgCnilWaGVpEbgHnMB6NZWPYBeKDQCtfPPAwXFcEuCxxWlEzENjbPi2j267u3jA32MzeWpRtu42O7GlF56dv-e0bwbLODZDNeFgobk Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: c0e363b1584848c1aecbe186cf91eec5 Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-AttachVolumeTestJSON-588294681 Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:47:14.636137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:47:14.637974 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/c0e363b1584848c1aecbe186cf91eec5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/c0e363b1584848c1aecbe186cf91eec5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_c0e363b1584848c1aecbe186cf91eec5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/c0e363b1584848c1aecbe186cf91eec5"}]}] Jul 11 14:47:14.637974 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-AttachVolumeTestJSON-588294681 Jul 11 14:47:14.637974 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: c0e363b1584848c1aecbe186cf91eec5 Jul 11 14:47:14.637974 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-AttachVolumeTestJSON-588294681 Jul 11 14:47:14.637974 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-AttachVolumeTestJSON-588294681-project Jul 11 14:47:14.637974 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:47:14.637974 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:47:14.637974 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 467992b53c4749858fd8b0ccd2db2393 Jul 11 14:47:14.637974 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-AttachVolumeTestJSON-588294681-project Jul 11 14:47:14.637974 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:47:14.637974 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance cb814dfe-6200-40ed-832f-df4face7a16e could not be found. Jul 11 14:47:14.637974 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': 'cb814dfe-6200-40ed-832f-df4face7a16e'} Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cb814dfe-6200-40ed-832f-df4face7a16e could not be found. Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:14.639873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cb814dfe-6200-40ed-832f-df4face7a16e could not be found. Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cb814dfe-6200-40ed-832f-df4face7a16e could not be found. Jul 11 14:47:14.642889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:14.647422 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-cd2e2013-ed47-4e00-94ad-9cf128ce77c8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Returning 404 to user: Instance cb814dfe-6200-40ed-832f-df4face7a16e could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:14.647422 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-cd2e2013-ed47-4e00-94ad-9cf128ce77c8 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "DELETE /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 404 len: 111 microversion: 2.1 time: 0.037922 Jul 11 14:47:14.647422 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 481/930] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:47:14 2022] DELETE /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:47:14.650027 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-51e13035-8ca0-4fdf-8909-dff5bbc7ca61 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:14.651711 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-51e13035-8ca0-4fdf-8909-dff5bbc7ca61 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: cb814dfe-6200-40ed-832f-df4face7a16e] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:14.658060 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-51e13035-8ca0-4fdf-8909-dff5bbc7ca61 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:14.659441 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-51e13035-8ca0-4fdf-8909-dff5bbc7ca61 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:14.660663 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-51e13035-8ca0-4fdf-8909-dff5bbc7ca61 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:14.661210 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-51e13035-8ca0-4fdf-8909-dff5bbc7ca61 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:14.688598 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-51e13035-8ca0-4fdf-8909-dff5bbc7ca61 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] HTTP exception thrown: Instance cb814dfe-6200-40ed-832f-df4face7a16e could not be found. Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-51e13035-8ca0-4fdf-8909-dff5bbc7ca61 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Request method failure captured: Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e HTTP/1.1 Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDZgOjz1xpcRYZZu-Kh_5N3i0wpyKor5piLH_ZAuxS7Y7F4dt1mCndOM5aBnPiEJue8x_XFxSfgCnilWaGVpEbgHnMB6NZWPYBeKDQCtfPPAwXFcEuCxxWlEzENjbPi2j267u3jA32MzeWpRtu42O7GlF56dv-e0bwbLODZDNeFgobk Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: c0e363b1584848c1aecbe186cf91eec5 Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-AttachVolumeTestJSON-588294681 Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:47:14.691943 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:47:14.693722 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/c0e363b1584848c1aecbe186cf91eec5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/c0e363b1584848c1aecbe186cf91eec5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_c0e363b1584848c1aecbe186cf91eec5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/c0e363b1584848c1aecbe186cf91eec5"}]}] Jul 11 14:47:14.693722 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-AttachVolumeTestJSON-588294681 Jul 11 14:47:14.693722 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: c0e363b1584848c1aecbe186cf91eec5 Jul 11 14:47:14.693722 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-AttachVolumeTestJSON-588294681 Jul 11 14:47:14.693722 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-AttachVolumeTestJSON-588294681-project Jul 11 14:47:14.693722 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:47:14.693722 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:47:14.693722 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 467992b53c4749858fd8b0ccd2db2393 Jul 11 14:47:14.693722 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-AttachVolumeTestJSON-588294681-project Jul 11 14:47:14.693722 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:47:14.693722 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance cb814dfe-6200-40ed-832f-df4face7a16e could not be found. Jul 11 14:47:14.693722 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': 'cb814dfe-6200-40ed-832f-df4face7a16e'} Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cb814dfe-6200-40ed-832f-df4face7a16e could not be found. Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:14.694680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:14.696255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:14.696255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:47:14.696255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:47:14.696255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:47:14.696255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:47:14.696255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:47:14.696255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:47:14.696255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cb814dfe-6200-40ed-832f-df4face7a16e could not be found. Jul 11 14:47:14.696255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:14.696255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:14.696255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:14.696255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:14.696255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:14.696255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:14.696255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:47:14.696255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:47:14.696255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cb814dfe-6200-40ed-832f-df4face7a16e could not be found. Jul 11 14:47:14.696255 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:14.697777 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-51e13035-8ca0-4fdf-8909-dff5bbc7ca61 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Returning 404 to user: Instance cb814dfe-6200-40ed-832f-df4face7a16e could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:14.698874 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-51e13035-8ca0-4fdf-8909-dff5bbc7ca61 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e" status: 404 len: 111 microversion: 2.1 time: 0.053786 Jul 11 14:47:14.700597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 450/931] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:14 2022] GET /compute/v2.1/servers/cb814dfe-6200-40ed-832f-df4face7a16e => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:47:14.967199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a95c2b9d-fe09-4fd7-8b2e-bec3218d75cb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:14.968257 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a95c2b9d-fe09-4fd7-8b2e-bec3218d75cb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:14.976454 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a95c2b9d-fe09-4fd7-8b2e-bec3218d75cb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:14.976997 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a95c2b9d-fe09-4fd7-8b2e-bec3218d75cb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:14.978273 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a95c2b9d-fe09-4fd7-8b2e-bec3218d75cb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:14.978595 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a95c2b9d-fe09-4fd7-8b2e-bec3218d75cb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:15.198023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a95c2b9d-fe09-4fd7-8b2e-bec3218d75cb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:15.208416 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a95c2b9d-fe09-4fd7-8b2e-bec3218d75cb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:15.210283 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a95c2b9d-fe09-4fd7-8b2e-bec3218d75cb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:15.211378 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a95c2b9d-fe09-4fd7-8b2e-bec3218d75cb tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1729 microversion: 2.20 time: 0.249251 Jul 11 14:47:15.211951 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 482/932] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:47:14 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1729 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:47:15.668955 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7d9c30a6-722f-4a2a-86bb-9b32a71b8f0c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:15.669901 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-7d9c30a6-722f-4a2a-86bb-9b32a71b8f0c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:15.684625 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7d9c30a6-722f-4a2a-86bb-9b32a71b8f0c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:15.685446 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7d9c30a6-722f-4a2a-86bb-9b32a71b8f0c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:15.686722 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7d9c30a6-722f-4a2a-86bb-9b32a71b8f0c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:15.687222 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7d9c30a6-722f-4a2a-86bb-9b32a71b8f0c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:15.804863 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.objects.instance [None req-7d9c30a6-722f-4a2a-86bb-9b32a71b8f0c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lazy-loading 'services' on Instance uuid 01762198-189f-4f97-84e5-4e0073a3879b {{(pid=130160) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:47:15.914754 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-edf40704-e0d1-41e6-84e7-255230019a2c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:15.915598 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-edf40704-e0d1-41e6-84e7-255230019a2c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:15.927790 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-edf40704-e0d1-41e6-84e7-255230019a2c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:15.928528 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-edf40704-e0d1-41e6-84e7-255230019a2c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:15.930279 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-edf40704-e0d1-41e6-84e7-255230019a2c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:15.931064 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-edf40704-e0d1-41e6-84e7-255230019a2c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:15.935159 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-7d9c30a6-722f-4a2a-86bb-9b32a71b8f0c tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "DELETE /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b/os-volume_attachments/8a7d6bfe-a5be-4a10-a855-683b92af94a5" status: 202 len: 0 microversion: 2.1 time: 0.272866 Jul 11 14:47:15.936268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 451/933] 10.222.0.72 () {58 vars in 1415 bytes} [Mon Jul 11 14:47:15 2022] DELETE /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b/os-volume_attachments/8a7d6bfe-a5be-4a10-a855-683b92af94a5 => generated 0 bytes in 276 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:47:15.949835 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-bab49e53-2764-4286-a96e-2dd4003d6b53 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:15.952277 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-bab49e53-2764-4286-a96e-2dd4003d6b53 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:15.961010 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bab49e53-2764-4286-a96e-2dd4003d6b53 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:15.961472 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bab49e53-2764-4286-a96e-2dd4003d6b53 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:15.962421 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bab49e53-2764-4286-a96e-2dd4003d6b53 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:15.962799 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-bab49e53-2764-4286-a96e-2dd4003d6b53 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:15.963539 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-edf40704-e0d1-41e6-84e7-255230019a2c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] HTTP exception thrown: Instance 62a90c58-0027-47e8-8909-44512060e08d could not be found. Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-edf40704-e0d1-41e6-84e7-255230019a2c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Request method failure captured: Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: DELETE /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d HTTP/1.1 Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZgOjz1xpcRYZZu-Kh_5N3i0wpyKor5piLH_ZAuxS7Y7F4dt1mCndOM5aBnPiEJue8x_XFxSfgCnilWaGVpEbgHnMB6NZWPYBeKDQCtfPPAwXFcEuCxxWlEzENjbPi2j267u3jA32MzeWpRtu42O7GlF56dv-e0bwbLODZDNeFgobk Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: c0e363b1584848c1aecbe186cf91eec5 Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-AttachVolumeTestJSON-588294681 Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:47:15.969193 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:47:15.971671 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/c0e363b1584848c1aecbe186cf91eec5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/c0e363b1584848c1aecbe186cf91eec5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_c0e363b1584848c1aecbe186cf91eec5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/c0e363b1584848c1aecbe186cf91eec5"}]}] Jul 11 14:47:15.971671 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-AttachVolumeTestJSON-588294681 Jul 11 14:47:15.971671 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: c0e363b1584848c1aecbe186cf91eec5 Jul 11 14:47:15.971671 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-AttachVolumeTestJSON-588294681 Jul 11 14:47:15.971671 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-AttachVolumeTestJSON-588294681-project Jul 11 14:47:15.971671 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:47:15.971671 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:47:15.971671 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 467992b53c4749858fd8b0ccd2db2393 Jul 11 14:47:15.971671 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-AttachVolumeTestJSON-588294681-project Jul 11 14:47:15.971671 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:47:15.971671 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance 62a90c58-0027-47e8-8909-44512060e08d could not be found. Jul 11 14:47:15.971671 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '62a90c58-0027-47e8-8909-44512060e08d'} Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 62a90c58-0027-47e8-8909-44512060e08d could not be found. Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:15.972804 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 62a90c58-0027-47e8-8909-44512060e08d could not be found. Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 62a90c58-0027-47e8-8909-44512060e08d could not be found. Jul 11 14:47:15.974203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:15.975868 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-edf40704-e0d1-41e6-84e7-255230019a2c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Returning 404 to user: Instance 62a90c58-0027-47e8-8909-44512060e08d could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:15.975868 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-edf40704-e0d1-41e6-84e7-255230019a2c tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "DELETE /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 404 len: 111 microversion: 2.1 time: 0.066170 Jul 11 14:47:15.975868 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 483/934] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:47:15 2022] DELETE /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 111 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 11 14:47:15.992901 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e9c7f4bb-8b0e-4222-85c1-c848c985baf5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:15.993719 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e9c7f4bb-8b0e-4222-85c1-c848c985baf5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] [instance: 62a90c58-0027-47e8-8909-44512060e08d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:16.006799 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-bab49e53-2764-4286-a96e-2dd4003d6b53 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.065056 Jul 11 14:47:16.007586 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 452/935] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:47:15 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:47:16.007706 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e9c7f4bb-8b0e-4222-85c1-c848c985baf5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:16.008447 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e9c7f4bb-8b0e-4222-85c1-c848c985baf5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:16.010570 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e9c7f4bb-8b0e-4222-85c1-c848c985baf5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:16.011530 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e9c7f4bb-8b0e-4222-85c1-c848c985baf5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:16.042083 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-e9c7f4bb-8b0e-4222-85c1-c848c985baf5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] HTTP exception thrown: Instance 62a90c58-0027-47e8-8909-44512060e08d could not be found. Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e9c7f4bb-8b0e-4222-85c1-c848c985baf5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Request method failure captured: Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d HTTP/1.1 Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZgOjz1xpcRYZZu-Kh_5N3i0wpyKor5piLH_ZAuxS7Y7F4dt1mCndOM5aBnPiEJue8x_XFxSfgCnilWaGVpEbgHnMB6NZWPYBeKDQCtfPPAwXFcEuCxxWlEzENjbPi2j267u3jA32MzeWpRtu42O7GlF56dv-e0bwbLODZDNeFgobk Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: c0e363b1584848c1aecbe186cf91eec5 Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-AttachVolumeTestJSON-588294681 Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:47:16.043739 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:47:16.045631 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/c0e363b1584848c1aecbe186cf91eec5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/c0e363b1584848c1aecbe186cf91eec5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_c0e363b1584848c1aecbe186cf91eec5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/c0e363b1584848c1aecbe186cf91eec5"}]}] Jul 11 14:47:16.045631 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-AttachVolumeTestJSON-588294681 Jul 11 14:47:16.045631 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: c0e363b1584848c1aecbe186cf91eec5 Jul 11 14:47:16.045631 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-AttachVolumeTestJSON-588294681 Jul 11 14:47:16.045631 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-AttachVolumeTestJSON-588294681-project Jul 11 14:47:16.045631 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:47:16.045631 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:47:16.045631 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 467992b53c4749858fd8b0ccd2db2393 Jul 11 14:47:16.045631 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-AttachVolumeTestJSON-588294681-project Jul 11 14:47:16.045631 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:47:16.045631 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance 62a90c58-0027-47e8-8909-44512060e08d could not be found. Jul 11 14:47:16.045631 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '62a90c58-0027-47e8-8909-44512060e08d'} Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 62a90c58-0027-47e8-8909-44512060e08d could not be found. Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:16.046580 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:16.047781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:16.047781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:47:16.047781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:47:16.047781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:47:16.047781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:47:16.047781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:47:16.047781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:47:16.047781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 62a90c58-0027-47e8-8909-44512060e08d could not be found. Jul 11 14:47:16.047781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:16.047781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:16.047781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:16.047781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:16.047781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:16.047781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:16.047781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:47:16.047781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:47:16.047781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 62a90c58-0027-47e8-8909-44512060e08d could not be found. Jul 11 14:47:16.047781 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:16.049707 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e9c7f4bb-8b0e-4222-85c1-c848c985baf5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] Returning 404 to user: Instance 62a90c58-0027-47e8-8909-44512060e08d could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:16.049707 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e9c7f4bb-8b0e-4222-85c1-c848c985baf5 tempest-AttachVolumeTestJSON-588294681 tempest-AttachVolumeTestJSON-588294681-project] 10.222.0.72 "GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d" status: 404 len: 111 microversion: 2.1 time: 0.062469 Jul 11 14:47:16.049707 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 484/936] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:15 2022] GET /compute/v2.1/servers/62a90c58-0027-47e8-8909-44512060e08d => generated 111 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 11 14:47:16.235119 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-07ad5c3f-d9bb-429f-ad00-a78b41df4fec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:16.236822 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-07ad5c3f-d9bb-429f-ad00-a78b41df4fec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:16.247797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-07ad5c3f-d9bb-429f-ad00-a78b41df4fec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:16.248518 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-07ad5c3f-d9bb-429f-ad00-a78b41df4fec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:16.249909 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-07ad5c3f-d9bb-429f-ad00-a78b41df4fec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:16.250758 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-07ad5c3f-d9bb-429f-ad00-a78b41df4fec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:16.502866 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-07ad5c3f-d9bb-429f-ad00-a78b41df4fec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:16.512401 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-07ad5c3f-d9bb-429f-ad00-a78b41df4fec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:16.514040 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-07ad5c3f-d9bb-429f-ad00-a78b41df4fec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:16.515621 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-07ad5c3f-d9bb-429f-ad00-a78b41df4fec tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1716 microversion: 2.20 time: 0.288169 Jul 11 14:47:16.516685 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 453/937] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:47:16 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1716 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:47:17.028512 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-53d270b5-57e0-4a84-824c-7e27b3799fdf tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:17.030337 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-53d270b5-57e0-4a84-824c-7e27b3799fdf tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:17.043360 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-53d270b5-57e0-4a84-824c-7e27b3799fdf tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:17.044152 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-53d270b5-57e0-4a84-824c-7e27b3799fdf tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:17.045575 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-53d270b5-57e0-4a84-824c-7e27b3799fdf tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:17.046721 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-53d270b5-57e0-4a84-824c-7e27b3799fdf tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:17.107675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-53d270b5-57e0-4a84-824c-7e27b3799fdf tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.087893 Jul 11 14:47:17.108684 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 485/938] 10.222.0.72 () {58 vars in 1301 bytes} [Mon Jul 11 14:47:17 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b/os-volume_attachments => generated 25 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Jul 11 14:47:17.187481 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f889cf5a-30ef-4292-8a4d-ba05395c4513 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:17.188199 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f889cf5a-30ef-4292-8a4d-ba05395c4513 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:17.196068 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f889cf5a-30ef-4292-8a4d-ba05395c4513 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:17.196598 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f889cf5a-30ef-4292-8a4d-ba05395c4513 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:17.197524 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f889cf5a-30ef-4292-8a4d-ba05395c4513 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:17.198137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f889cf5a-30ef-4292-8a4d-ba05395c4513 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:17.235015 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f889cf5a-30ef-4292-8a4d-ba05395c4513 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Going to try to terminate instance {{(pid=130160) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:47:17.359469 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f889cf5a-30ef-4292-8a4d-ba05395c4513 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "DELETE /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 204 len: 0 microversion: 2.1 time: 0.175675 Jul 11 14:47:17.364729 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 454/939] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:47:17 2022] DELETE /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 11 14:47:17.371757 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-ea28ea97-a3cd-4e5e-ba6c-f0aa987e8e6d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:17.372998 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-ea28ea97-a3cd-4e5e-ba6c-f0aa987e8e6d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:17.383368 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ea28ea97-a3cd-4e5e-ba6c-f0aa987e8e6d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:17.384292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ea28ea97-a3cd-4e5e-ba6c-f0aa987e8e6d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:17.385467 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ea28ea97-a3cd-4e5e-ba6c-f0aa987e8e6d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:17.386280 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ea28ea97-a3cd-4e5e-ba6c-f0aa987e8e6d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:17.735568 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-ea28ea97-a3cd-4e5e-ba6c-f0aa987e8e6d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:17.745248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-ea28ea97-a3cd-4e5e-ba6c-f0aa987e8e6d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 200 len: 1682 microversion: 2.1 time: 0.378893 Jul 11 14:47:17.746000 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 486/940] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:17 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 1682 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:47:18.767002 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2531803b-555a-4707-b13d-8976c1efd380 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:18.768519 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2531803b-555a-4707-b13d-8976c1efd380 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:18.779826 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2531803b-555a-4707-b13d-8976c1efd380 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:18.780542 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2531803b-555a-4707-b13d-8976c1efd380 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:18.781552 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2531803b-555a-4707-b13d-8976c1efd380 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:18.782583 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2531803b-555a-4707-b13d-8976c1efd380 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:19.489805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2531803b-555a-4707-b13d-8976c1efd380 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:19.501363 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2531803b-555a-4707-b13d-8976c1efd380 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 200 len: 1682 microversion: 2.1 time: 0.740780 Jul 11 14:47:19.529401 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 455/941] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:18 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 1682 bytes in 770 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:47:20.538156 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-7dc9f0fe-4391-49fd-8636-0c3be82641c8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:20.546291 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-7dc9f0fe-4391-49fd-8636-0c3be82641c8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:20.589140 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7dc9f0fe-4391-49fd-8636-0c3be82641c8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:20.594388 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7dc9f0fe-4391-49fd-8636-0c3be82641c8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:20.598563 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7dc9f0fe-4391-49fd-8636-0c3be82641c8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:20.601645 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7dc9f0fe-4391-49fd-8636-0c3be82641c8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:20.858390 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7dc9f0fe-4391-49fd-8636-0c3be82641c8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '3ae56fd1172f4bf8806eefc6e8877fef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1baa6ea2e19747d1af1e1042c362895d', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:20.871741 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=100 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:47:20.882889 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-7dc9f0fe-4391-49fd-8636-0c3be82641c8 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 200 len: 1614 microversion: 2.1 time: 0.358970 Jul 11 14:47:20.883686 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 487/942] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:20 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 1614 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:47:20.919322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-0ddcb27a-c651-46e0-98ca-dba6fbaa6656 req-b047eec8-ec8b-48ae-ac6f-171db63e9b93 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "01762198-189f-4f97-84e5-4e0073a3879b", "name": "network-vif-deleted", "tag": "c6732da2-2010-4021-a69d-771c91c0bcac"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:47:20.929678 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-0ddcb27a-c651-46e0-98ca-dba6fbaa6656 req-b047eec8-ec8b-48ae-ac6f-171db63e9b93 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:20.930381 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-0ddcb27a-c651-46e0-98ca-dba6fbaa6656 req-b047eec8-ec8b-48ae-ac6f-171db63e9b93 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:20.945145 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:47:20.950480 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-0ddcb27a-c651-46e0-98ca-dba6fbaa6656 req-b047eec8-ec8b-48ae-ac6f-171db63e9b93 service nova] Creating event network-vif-deleted:c6732da2-2010-4021-a69d-771c91c0bcac for instance 01762198-189f-4f97-84e5-4e0073a3879b on hv-cinder-maste Jul 11 14:47:20.965217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-0ddcb27a-c651-46e0-98ca-dba6fbaa6656 req-b047eec8-ec8b-48ae-ac6f-171db63e9b93 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051124 Jul 11 14:47:20.982406 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 456/943] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:47:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:47:21.906275 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8609a40a-f358-4967-8b53-9840cef973e2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:21.907503 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8609a40a-f358-4967-8b53-9840cef973e2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:21.926154 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8609a40a-f358-4967-8b53-9840cef973e2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:21.927014 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8609a40a-f358-4967-8b53-9840cef973e2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:21.928463 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8609a40a-f358-4967-8b53-9840cef973e2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:21.929258 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8609a40a-f358-4967-8b53-9840cef973e2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:21.971657 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-8609a40a-f358-4967-8b53-9840cef973e2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] HTTP exception thrown: Instance 01762198-189f-4f97-84e5-4e0073a3879b could not be found. Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8609a40a-f358-4967-8b53-9840cef973e2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Request method failure captured: Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b HTTP/1.1 Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZbQUJdl7M18phk4FM9-GGzi8XbGx24YhEK9KsOfI165n5v27J9wq7wzJjs1m7qkLCobj_5JRaSFQlNcVRL5jfwSqdoh1nodKIJxEOpM9gSJP97YpvVDnXLPsMNjwZQwG04QTf0uG5zLXZ7eKV_ivBXCJkCsdmhsPnE9ln-9M7wHs4 Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:47:21.974797 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:47:21.977698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}] Jul 11 14:47:21.977698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:21.977698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:47:21.977698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:21.977698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:47:21.977698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:47:21.977698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:47:21.977698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 3ae56fd1172f4bf8806eefc6e8877fef Jul 11 14:47:21.977698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:47:21.977698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:47:21.977698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance 01762198-189f-4f97-84e5-4e0073a3879b could not be found. Jul 11 14:47:21.977698 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '01762198-189f-4f97-84e5-4e0073a3879b'} Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 01762198-189f-4f97-84e5-4e0073a3879b could not be found. Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:21.979381 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:21.981770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:21.981770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:47:21.981770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:47:21.981770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:47:21.981770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:47:21.981770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:47:21.981770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:47:21.981770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 01762198-189f-4f97-84e5-4e0073a3879b could not be found. Jul 11 14:47:21.981770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:21.981770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:21.981770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:21.981770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:21.981770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:21.981770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:21.981770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:47:21.981770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:47:21.981770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 01762198-189f-4f97-84e5-4e0073a3879b could not be found. Jul 11 14:47:21.981770 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:21.985422 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8609a40a-f358-4967-8b53-9840cef973e2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Returning 404 to user: Instance 01762198-189f-4f97-84e5-4e0073a3879b could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:21.987376 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8609a40a-f358-4967-8b53-9840cef973e2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 404 len: 111 microversion: 2.1 time: 0.088789 Jul 11 14:47:21.994061 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 488/944] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:21 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 111 bytes in 97 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:47:22.025634 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-39855c22-0d9b-488c-bd06-c4bfcf557501 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '.version_select at 0x7f592b7639d0>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:22.069067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-39855c22-0d9b-488c-bd06-c4bfcf557501 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1469998488" status: 202 len: 0 microversion: 2.1 time: 0.072128 Jul 11 14:47:22.077199 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 457/945] 10.222.0.72 () {58 vars in 1220 bytes} [Mon Jul 11 14:47:21 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1469998488 => generated 0 bytes in 75 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:47:24.383055 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-4d3834d5-8471-4755-a464-5a518a1d9e72 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:24.383917 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-4d3834d5-8471-4755-a464-5a518a1d9e72 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:24.396090 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4d3834d5-8471-4755-a464-5a518a1d9e72 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:24.396858 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4d3834d5-8471-4755-a464-5a518a1d9e72 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:24.398191 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4d3834d5-8471-4755-a464-5a518a1d9e72 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:24.398969 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4d3834d5-8471-4755-a464-5a518a1d9e72 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:24.424765 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-4d3834d5-8471-4755-a464-5a518a1d9e72 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] HTTP exception thrown: Instance 01762198-189f-4f97-84e5-4e0073a3879b could not be found. Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-4d3834d5-8471-4755-a464-5a518a1d9e72 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Request method failure captured: Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: DELETE /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b HTTP/1.1 Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZbQUJdl7M18phk4FM9-GGzi8XbGx24YhEK9KsOfI165n5v27J9wq7wzJjs1m7qkLCobj_5JRaSFQlNcVRL5jfwSqdoh1nodKIJxEOpM9gSJP97YpvVDnXLPsMNjwZQwG04QTf0uG5zLXZ7eKV_ivBXCJkCsdmhsPnE9ln-9M7wHs4 Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:47:24.427782 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:47:24.430918 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}] Jul 11 14:47:24.430918 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:24.430918 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:47:24.430918 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:24.430918 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:47:24.430918 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:47:24.430918 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:47:24.430918 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 3ae56fd1172f4bf8806eefc6e8877fef Jul 11 14:47:24.430918 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:47:24.430918 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:47:24.430918 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance 01762198-189f-4f97-84e5-4e0073a3879b could not be found. Jul 11 14:47:24.430918 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '01762198-189f-4f97-84e5-4e0073a3879b'} Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 01762198-189f-4f97-84e5-4e0073a3879b could not be found. Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:24.436162 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 01762198-189f-4f97-84e5-4e0073a3879b could not be found. Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 01762198-189f-4f97-84e5-4e0073a3879b could not be found. Jul 11 14:47:24.439078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:24.442093 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-4d3834d5-8471-4755-a464-5a518a1d9e72 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Returning 404 to user: Instance 01762198-189f-4f97-84e5-4e0073a3879b could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:24.442093 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-4d3834d5-8471-4755-a464-5a518a1d9e72 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "DELETE /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 404 len: 111 microversion: 2.1 time: 0.054476 Jul 11 14:47:24.442093 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 489/946] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:47:24 2022] DELETE /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 11 14:47:24.442894 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e65b90dc-a0c5-4071-a55d-581bd35a824d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:24.444214 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-e65b90dc-a0c5-4071-a55d-581bd35a824d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 01762198-189f-4f97-84e5-4e0073a3879b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:24.456412 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e65b90dc-a0c5-4071-a55d-581bd35a824d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:24.456964 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e65b90dc-a0c5-4071-a55d-581bd35a824d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:24.458705 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e65b90dc-a0c5-4071-a55d-581bd35a824d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:24.459457 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e65b90dc-a0c5-4071-a55d-581bd35a824d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:24.482695 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-e65b90dc-a0c5-4071-a55d-581bd35a824d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] HTTP exception thrown: Instance 01762198-189f-4f97-84e5-4e0073a3879b could not be found. Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e65b90dc-a0c5-4071-a55d-581bd35a824d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Request method failure captured: Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b HTTP/1.1 Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDZbQUJdl7M18phk4FM9-GGzi8XbGx24YhEK9KsOfI165n5v27J9wq7wzJjs1m7qkLCobj_5JRaSFQlNcVRL5jfwSqdoh1nodKIJxEOpM9gSJP97YpvVDnXLPsMNjwZQwG04QTf0uG5zLXZ7eKV_ivBXCJkCsdmhsPnE9ln-9M7wHs4 Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:47:24.485071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:47:24.487784 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}] Jul 11 14:47:24.487784 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:24.487784 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:47:24.487784 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:24.487784 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:47:24.487784 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:47:24.487784 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:47:24.487784 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 3ae56fd1172f4bf8806eefc6e8877fef Jul 11 14:47:24.487784 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:47:24.487784 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:47:24.487784 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance 01762198-189f-4f97-84e5-4e0073a3879b could not be found. Jul 11 14:47:24.487784 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': '01762198-189f-4f97-84e5-4e0073a3879b'} Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 01762198-189f-4f97-84e5-4e0073a3879b could not be found. Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:24.489462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:24.492217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:24.492217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:47:24.492217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:47:24.492217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:47:24.492217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:47:24.492217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:47:24.492217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:47:24.492217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 01762198-189f-4f97-84e5-4e0073a3879b could not be found. Jul 11 14:47:24.492217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:24.492217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:24.492217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:24.492217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:24.492217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:24.492217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:24.492217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:47:24.492217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:47:24.492217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 01762198-189f-4f97-84e5-4e0073a3879b could not be found. Jul 11 14:47:24.492217 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:24.494589 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e65b90dc-a0c5-4071-a55d-581bd35a824d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Returning 404 to user: Instance 01762198-189f-4f97-84e5-4e0073a3879b could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:24.494589 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e65b90dc-a0c5-4071-a55d-581bd35a824d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b" status: 404 len: 111 microversion: 2.1 time: 0.050750 Jul 11 14:47:24.494589 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 458/947] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:24 2022] GET /compute/v2.1/servers/01762198-189f-4f97-84e5-4e0073a3879b => generated 111 bytes in 53 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:47:25.716822 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-84e602a2-ec3c-404d-8683-d0fc95a63c6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:25.717737 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-84e602a2-ec3c-404d-8683-d0fc95a63c6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:25.722655 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=65,nova_api:DELETE=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:47:25.733451 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84e602a2-ec3c-404d-8683-d0fc95a63c6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:25.734279 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84e602a2-ec3c-404d-8683-d0fc95a63c6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:25.735826 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84e602a2-ec3c-404d-8683-d0fc95a63c6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:25.737237 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84e602a2-ec3c-404d-8683-d0fc95a63c6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:25.756057 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-84e602a2-ec3c-404d-8683-d0fc95a63c6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] HTTP exception thrown: Instance 1f161996-6f14-46a9-9200-c8763cb67e21 could not be found. Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-84e602a2-ec3c-404d-8683-d0fc95a63c6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Request method failure captured: Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: DELETE /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 HTTP/1.1 Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZbQUJdl7M18phk4FM9-GGzi8XbGx24YhEK9KsOfI165n5v27J9wq7wzJjs1m7qkLCobj_5JRaSFQlNcVRL5jfwSqdoh1nodKIJxEOpM9gSJP97YpvVDnXLPsMNjwZQwG04QTf0uG5zLXZ7eKV_ivBXCJkCsdmhsPnE9ln-9M7wHs4 Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:47:25.757664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:47:25.759439 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}] Jul 11 14:47:25.759439 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:25.759439 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:47:25.759439 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:25.759439 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:47:25.759439 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:47:25.759439 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:47:25.759439 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 3ae56fd1172f4bf8806eefc6e8877fef Jul 11 14:47:25.759439 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:47:25.759439 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:47:25.759439 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance 1f161996-6f14-46a9-9200-c8763cb67e21 could not be found. Jul 11 14:47:25.759439 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '1f161996-6f14-46a9-9200-c8763cb67e21'} Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1f161996-6f14-46a9-9200-c8763cb67e21 could not be found. Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:25.760458 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1f161996-6f14-46a9-9200-c8763cb67e21 could not be found. Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1f161996-6f14-46a9-9200-c8763cb67e21 could not be found. Jul 11 14:47:25.761964 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.765192 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-84e602a2-ec3c-404d-8683-d0fc95a63c6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Returning 404 to user: Instance 1f161996-6f14-46a9-9200-c8763cb67e21 could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:25.765192 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-84e602a2-ec3c-404d-8683-d0fc95a63c6e tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "DELETE /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 404 len: 111 microversion: 2.1 time: 0.050694 Jul 11 14:47:25.766376 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 490/948] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:47:25 2022] DELETE /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 111 bytes in 58 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:47:25.772830 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-10b9a5b3-1254-4b30-8fc9-adc4c8690ce0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:25.773598 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-10b9a5b3-1254-4b30-8fc9-adc4c8690ce0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 1f161996-6f14-46a9-9200-c8763cb67e21] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:25.781512 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-10b9a5b3-1254-4b30-8fc9-adc4c8690ce0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:25.782110 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-10b9a5b3-1254-4b30-8fc9-adc4c8690ce0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:25.783680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-10b9a5b3-1254-4b30-8fc9-adc4c8690ce0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:25.784310 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-10b9a5b3-1254-4b30-8fc9-adc4c8690ce0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-10b9a5b3-1254-4b30-8fc9-adc4c8690ce0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] HTTP exception thrown: Instance 1f161996-6f14-46a9-9200-c8763cb67e21 could not be found. Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-10b9a5b3-1254-4b30-8fc9-adc4c8690ce0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Request method failure captured: Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 HTTP/1.1 Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDZbQUJdl7M18phk4FM9-GGzi8XbGx24YhEK9KsOfI165n5v27J9wq7wzJjs1m7qkLCobj_5JRaSFQlNcVRL5jfwSqdoh1nodKIJxEOpM9gSJP97YpvVDnXLPsMNjwZQwG04QTf0uG5zLXZ7eKV_ivBXCJkCsdmhsPnE9ln-9M7wHs4 Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:47:25.804915 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:47:25.806322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}] Jul 11 14:47:25.806322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:25.806322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:47:25.806322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:25.806322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:47:25.806322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:47:25.806322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:47:25.806322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 3ae56fd1172f4bf8806eefc6e8877fef Jul 11 14:47:25.806322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:47:25.806322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:47:25.806322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance 1f161996-6f14-46a9-9200-c8763cb67e21 could not be found. Jul 11 14:47:25.806322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': '1f161996-6f14-46a9-9200-c8763cb67e21'} Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1f161996-6f14-46a9-9200-c8763cb67e21 could not be found. Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:25.807144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:25.808579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:25.808579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:47:25.808579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:47:25.808579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:47:25.808579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:47:25.808579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:47:25.808579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:47:25.808579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1f161996-6f14-46a9-9200-c8763cb67e21 could not be found. Jul 11 14:47:25.808579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.808579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:25.808579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.808579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:25.808579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:25.808579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:25.808579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:47:25.808579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:47:25.808579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1f161996-6f14-46a9-9200-c8763cb67e21 could not be found. Jul 11 14:47:25.808579 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.809787 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-10b9a5b3-1254-4b30-8fc9-adc4c8690ce0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Returning 404 to user: Instance 1f161996-6f14-46a9-9200-c8763cb67e21 could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:25.809787 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-10b9a5b3-1254-4b30-8fc9-adc4c8690ce0 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21" status: 404 len: 111 microversion: 2.1 time: 0.039680 Jul 11 14:47:25.809787 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 459/949] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:25 2022] GET /compute/v2.1/servers/1f161996-6f14-46a9-9200-c8763cb67e21 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 11 14:47:25.816807 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3dad770e-58d8-49e6-8022-aebff3bb097f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:25.817315 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3dad770e-58d8-49e6-8022-aebff3bb097f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:25.824679 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3dad770e-58d8-49e6-8022-aebff3bb097f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:25.825137 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3dad770e-58d8-49e6-8022-aebff3bb097f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:25.826136 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3dad770e-58d8-49e6-8022-aebff3bb097f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:25.826823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3dad770e-58d8-49e6-8022-aebff3bb097f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:25.842813 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-3dad770e-58d8-49e6-8022-aebff3bb097f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] HTTP exception thrown: Instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 could not be found. Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3dad770e-58d8-49e6-8022-aebff3bb097f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Request method failure captured: Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: DELETE /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 HTTP/1.1 Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZbQUJdl7M18phk4FM9-GGzi8XbGx24YhEK9KsOfI165n5v27J9wq7wzJjs1m7qkLCobj_5JRaSFQlNcVRL5jfwSqdoh1nodKIJxEOpM9gSJP97YpvVDnXLPsMNjwZQwG04QTf0uG5zLXZ7eKV_ivBXCJkCsdmhsPnE9ln-9M7wHs4 Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:47:25.844437 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:47:25.846935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}] Jul 11 14:47:25.846935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:25.846935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:47:25.846935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:25.846935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:47:25.846935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:47:25.846935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:47:25.846935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 3ae56fd1172f4bf8806eefc6e8877fef Jul 11 14:47:25.846935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:47:25.846935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:47:25.846935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 could not be found. Jul 11 14:47:25.846935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '7a49d1ca-7491-41e0-b0d8-37c92b9d3562'} Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 could not be found. Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:25.848199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 could not be found. Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 could not be found. Jul 11 14:47:25.850540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3dad770e-58d8-49e6-8022-aebff3bb097f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Returning 404 to user: Instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:25.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3dad770e-58d8-49e6-8022-aebff3bb097f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "DELETE /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 404 len: 111 microversion: 2.1 time: 0.033582 Jul 11 14:47:25.852870 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 491/950] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:47:25 2022] DELETE /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 11 14:47:25.862423 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-399220fb-9fd9-46ac-bbe8-cc3cdb1a798f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:25.863157 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-399220fb-9fd9-46ac-bbe8-cc3cdb1a798f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: 7a49d1ca-7491-41e0-b0d8-37c92b9d3562] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:25.870045 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-399220fb-9fd9-46ac-bbe8-cc3cdb1a798f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:25.870709 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-399220fb-9fd9-46ac-bbe8-cc3cdb1a798f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:25.871632 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-399220fb-9fd9-46ac-bbe8-cc3cdb1a798f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:25.872298 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-399220fb-9fd9-46ac-bbe8-cc3cdb1a798f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:25.888138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-399220fb-9fd9-46ac-bbe8-cc3cdb1a798f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] HTTP exception thrown: Instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 could not be found. Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-399220fb-9fd9-46ac-bbe8-cc3cdb1a798f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Request method failure captured: Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 HTTP/1.1 Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDZbQUJdl7M18phk4FM9-GGzi8XbGx24YhEK9KsOfI165n5v27J9wq7wzJjs1m7qkLCobj_5JRaSFQlNcVRL5jfwSqdoh1nodKIJxEOpM9gSJP97YpvVDnXLPsMNjwZQwG04QTf0uG5zLXZ7eKV_ivBXCJkCsdmhsPnE9ln-9M7wHs4 Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:47:25.889921 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:47:25.892864 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}] Jul 11 14:47:25.892864 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:25.892864 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:47:25.892864 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:25.892864 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:47:25.892864 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:47:25.892864 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:47:25.892864 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 3ae56fd1172f4bf8806eefc6e8877fef Jul 11 14:47:25.892864 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:47:25.892864 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:47:25.892864 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 could not be found. Jul 11 14:47:25.892864 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': '7a49d1ca-7491-41e0-b0d8-37c92b9d3562'} Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 could not be found. Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:25.894549 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:25.903574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:25.903574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:47:25.903574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:47:25.903574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:47:25.903574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:47:25.903574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:47:25.903574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:47:25.903574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 could not be found. Jul 11 14:47:25.903574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.903574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:25.903574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.903574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:25.903574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:25.903574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:25.903574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:47:25.903574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:47:25.903574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 could not be found. Jul 11 14:47:25.903574 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:25.906231 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-399220fb-9fd9-46ac-bbe8-cc3cdb1a798f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Returning 404 to user: Instance 7a49d1ca-7491-41e0-b0d8-37c92b9d3562 could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:25.906231 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-399220fb-9fd9-46ac-bbe8-cc3cdb1a798f tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562" status: 404 len: 111 microversion: 2.1 time: 0.038132 Jul 11 14:47:25.906231 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 460/951] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:25 2022] GET /compute/v2.1/servers/7a49d1ca-7491-41e0-b0d8-37c92b9d3562 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:47:27.110757 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-bc4a8810-4581-406e-ac5a-3195a0f749a2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:27.112222 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-bc4a8810-4581-406e-ac5a-3195a0f749a2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:27.122730 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bc4a8810-4581-406e-ac5a-3195a0f749a2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:27.123386 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bc4a8810-4581-406e-ac5a-3195a0f749a2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:27.124944 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bc4a8810-4581-406e-ac5a-3195a0f749a2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:27.125688 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bc4a8810-4581-406e-ac5a-3195a0f749a2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:27.145801 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-bc4a8810-4581-406e-ac5a-3195a0f749a2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] HTTP exception thrown: Instance bd855cba-6708-47de-826b-00c5a74b23f2 could not be found. Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-bc4a8810-4581-406e-ac5a-3195a0f749a2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Request method failure captured: Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: DELETE /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 HTTP/1.1 Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZbQUJdl7M18phk4FM9-GGzi8XbGx24YhEK9KsOfI165n5v27J9wq7wzJjs1m7qkLCobj_5JRaSFQlNcVRL5jfwSqdoh1nodKIJxEOpM9gSJP97YpvVDnXLPsMNjwZQwG04QTf0uG5zLXZ7eKV_ivBXCJkCsdmhsPnE9ln-9M7wHs4 Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:47:27.148079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:47:27.151031 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}] Jul 11 14:47:27.151031 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:27.151031 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:47:27.151031 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:27.151031 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:47:27.151031 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:47:27.151031 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:47:27.151031 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 3ae56fd1172f4bf8806eefc6e8877fef Jul 11 14:47:27.151031 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:47:27.151031 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:47:27.151031 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance bd855cba-6708-47de-826b-00c5a74b23f2 could not be found. Jul 11 14:47:27.151031 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': 'bd855cba-6708-47de-826b-00c5a74b23f2'} Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bd855cba-6708-47de-826b-00c5a74b23f2 could not be found. Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:27.152509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bd855cba-6708-47de-826b-00c5a74b23f2 could not be found. Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bd855cba-6708-47de-826b-00c5a74b23f2 could not be found. Jul 11 14:47:27.156981 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:27.161626 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-bc4a8810-4581-406e-ac5a-3195a0f749a2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Returning 404 to user: Instance bd855cba-6708-47de-826b-00c5a74b23f2 could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:27.161626 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-bc4a8810-4581-406e-ac5a-3195a0f749a2 tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "DELETE /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 404 len: 111 microversion: 2.1 time: 0.045473 Jul 11 14:47:27.161626 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 492/952] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:47:27 2022] DELETE /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:47:27.162311 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3bf1d2d4-44ff-49e9-8119-e9e133be043d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:27.162311 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-3bf1d2d4-44ff-49e9-8119-e9e133be043d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] [instance: bd855cba-6708-47de-826b-00c5a74b23f2] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:27.169422 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3bf1d2d4-44ff-49e9-8119-e9e133be043d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:27.169726 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3bf1d2d4-44ff-49e9-8119-e9e133be043d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:27.170502 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3bf1d2d4-44ff-49e9-8119-e9e133be043d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:27.171487 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3bf1d2d4-44ff-49e9-8119-e9e133be043d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:27.187276 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-3bf1d2d4-44ff-49e9-8119-e9e133be043d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] HTTP exception thrown: Instance bd855cba-6708-47de-826b-00c5a74b23f2 could not be found. Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3bf1d2d4-44ff-49e9-8119-e9e133be043d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Request method failure captured: Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 HTTP/1.1 Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDZbQUJdl7M18phk4FM9-GGzi8XbGx24YhEK9KsOfI165n5v27J9wq7wzJjs1m7qkLCobj_5JRaSFQlNcVRL5jfwSqdoh1nodKIJxEOpM9gSJP97YpvVDnXLPsMNjwZQwG04QTf0uG5zLXZ7eKV_ivBXCJkCsdmhsPnE9ln-9M7wHs4 Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:47:27.189184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:47:27.192018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_1baa6ea2e19747d1af1e1042c362895d"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/1baa6ea2e19747d1af1e1042c362895d"}]}] Jul 11 14:47:27.192018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:27.192018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: 1baa6ea2e19747d1af1e1042c362895d Jul 11 14:47:27.192018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-428636144 Jul 11 14:47:27.192018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:47:27.192018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:47:27.192018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:47:27.192018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 3ae56fd1172f4bf8806eefc6e8877fef Jul 11 14:47:27.192018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-AttachVolumeNegativeTest-428636144-project Jul 11 14:47:27.192018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:47:27.192018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance bd855cba-6708-47de-826b-00c5a74b23f2 could not be found. Jul 11 14:47:27.192018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': 'bd855cba-6708-47de-826b-00c5a74b23f2'} Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bd855cba-6708-47de-826b-00c5a74b23f2 could not be found. Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:27.193137 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:27.195233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:27.195233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:47:27.195233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:47:27.195233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:47:27.195233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:47:27.195233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:47:27.195233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:47:27.195233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bd855cba-6708-47de-826b-00c5a74b23f2 could not be found. Jul 11 14:47:27.195233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:27.195233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:27.195233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:27.195233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:27.195233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:27.195233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:27.195233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:47:27.195233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:47:27.195233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bd855cba-6708-47de-826b-00c5a74b23f2 could not be found. Jul 11 14:47:27.195233 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:27.197297 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3bf1d2d4-44ff-49e9-8119-e9e133be043d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] Returning 404 to user: Instance bd855cba-6708-47de-826b-00c5a74b23f2 could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:27.197297 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-3bf1d2d4-44ff-49e9-8119-e9e133be043d tempest-AttachVolumeNegativeTest-428636144 tempest-AttachVolumeNegativeTest-428636144-project] 10.222.0.72 "GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2" status: 404 len: 111 microversion: 2.1 time: 0.034116 Jul 11 14:47:27.197297 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 461/953] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:27 2022] GET /compute/v2.1/servers/bd855cba-6708-47de-826b-00c5a74b23f2 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 11 14:47:34.197545 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-6011c76e-6521-45a1-9c87-4cd9cb0a3a12 tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] Action: 'create', calling method: .version_select at 0x7f592af37a60>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1007184453", "metadata": {"Type": "work"}}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-6011c76e-6521-45a1-9c87-4cd9cb0a3a12 tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] Request method failure captured: Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 122 Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDgFLT1oXLsBaNjknWAIzrQn0Mli7JeYXx0phdGxPoDsISV14y9csZluXDN3sCK6CL0i_6R8A48YD2G7_DL-vrbqx7Dg9tSLxnql8JraNNNdFUKmjN0Xf66l9kBStt9vw8LAG3JFwWkCHWkTzH2xqY7hFKuI4LrrFtj4LHTj-iR-1fk Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: ab6df275ee124e1a90efacdf16c6fa2e Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-VolumesNegativeTest-559576417 Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:47:34.217248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:47:34.219942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/ab6df275ee124e1a90efacdf16c6fa2e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/ab6df275ee124e1a90efacdf16c6fa2e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_ab6df275ee124e1a90efacdf16c6fa2e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/ab6df275ee124e1a90efacdf16c6fa2e"}]}] Jul 11 14:47:34.219942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-VolumesNegativeTest-559576417 Jul 11 14:47:34.219942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: ab6df275ee124e1a90efacdf16c6fa2e Jul 11 14:47:34.219942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-VolumesNegativeTest-559576417 Jul 11 14:47:34.219942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-VolumesNegativeTest-559576417-project Jul 11 14:47:34.219942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:47:34.219942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:47:34.219942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 6352e8010a3d4163a5b9c3fd6d68b37b Jul 11 14:47:34.219942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-VolumesNegativeTest-559576417-project Jul 11 14:47:34.219942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Jul 11 14:47:34.219942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1007184453", "metadata": {"Type": "work"}}} Jul 11 14:47:34.219942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: Jul 11 14:47:34.219942 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1007184453', 'metadata': {'Type': 'work'}}}} Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise error Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi '#$%' Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:34.222305 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 11 14:47:34.225506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 11 14:47:34.228178 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Jul 11 14:47:34.228178 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.228178 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-6011c76e-6521-45a1-9c87-4cd9cb0a3a12 tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:34.228178 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-6011c76e-6521-45a1-9c87-4cd9cb0a3a12 tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] 10.222.0.72 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.153975 Jul 11 14:47:34.228178 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 493/954] 10.222.0.72 () {60 vars in 1155 bytes} [Mon Jul 11 14:47:34 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 156 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 11 14:47:34.233253 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-1166c7fb-7e4a-40a8-bd78-5d6a73c6f0fb tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] Action: 'create', calling method: .version_select at 0x7f592b7484c0>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-186588960", "metadata": {"Type": "work"}}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-1166c7fb-7e4a-40a8-bd78-5d6a73c6f0fb tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] Request method failure captured: Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 119 Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDgFLT1oXLsBaNjknWAIzrQn0Mli7JeYXx0phdGxPoDsISV14y9csZluXDN3sCK6CL0i_6R8A48YD2G7_DL-vrbqx7Dg9tSLxnql8JraNNNdFUKmjN0Xf66l9kBStt9vw8LAG3JFwWkCHWkTzH2xqY7hFKuI4LrrFtj4LHTj-iR-1fk Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: ab6df275ee124e1a90efacdf16c6fa2e Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-VolumesNegativeTest-559576417 Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:47:34.237254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:47:34.239797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/ab6df275ee124e1a90efacdf16c6fa2e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/ab6df275ee124e1a90efacdf16c6fa2e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_ab6df275ee124e1a90efacdf16c6fa2e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/ab6df275ee124e1a90efacdf16c6fa2e"}]}] Jul 11 14:47:34.239797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-VolumesNegativeTest-559576417 Jul 11 14:47:34.239797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: ab6df275ee124e1a90efacdf16c6fa2e Jul 11 14:47:34.239797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-VolumesNegativeTest-559576417 Jul 11 14:47:34.239797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-VolumesNegativeTest-559576417-project Jul 11 14:47:34.239797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:47:34.239797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:47:34.239797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 6352e8010a3d4163a5b9c3fd6d68b37b Jul 11 14:47:34.239797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-VolumesNegativeTest-559576417-project Jul 11 14:47:34.239797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Jul 11 14:47:34.239797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-186588960", "metadata": {"Type": "work"}}} Jul 11 14:47:34.239797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: Jul 11 14:47:34.239797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-186588960', 'metadata': {'Type': 'work'}}}} Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise error Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi '0' Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:34.241558 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 11 14:47:34.244355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 11 14:47:34.246704 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Jul 11 14:47:34.246704 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.246704 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-1166c7fb-7e4a-40a8-bd78-5d6a73c6f0fb tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:34.246704 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-1166c7fb-7e4a-40a8-bd78-5d6a73c6f0fb tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] 10.222.0.72 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.009757 Jul 11 14:47:34.246704 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 462/955] 10.222.0.72 () {60 vars in 1155 bytes} [Mon Jul 11 14:47:34 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 11 14:47:34.255130 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-55371c43-8bc8-499a-85e4-a85cb1c607b4 tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] Action: 'create', calling method: .version_select at 0x7f5930269f70>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-223518008", "metadata": {"Type": "work"}}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-55371c43-8bc8-499a-85e4-a85cb1c607b4 tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] Request method failure captured: Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 118 Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDgFLT1oXLsBaNjknWAIzrQn0Mli7JeYXx0phdGxPoDsISV14y9csZluXDN3sCK6CL0i_6R8A48YD2G7_DL-vrbqx7Dg9tSLxnql8JraNNNdFUKmjN0Xf66l9kBStt9vw8LAG3JFwWkCHWkTzH2xqY7hFKuI4LrrFtj4LHTj-iR-1fk Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: ab6df275ee124e1a90efacdf16c6fa2e Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-VolumesNegativeTest-559576417 Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:47:34.261281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:47:34.264085 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/ab6df275ee124e1a90efacdf16c6fa2e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/ab6df275ee124e1a90efacdf16c6fa2e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_ab6df275ee124e1a90efacdf16c6fa2e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/ab6df275ee124e1a90efacdf16c6fa2e"}]}] Jul 11 14:47:34.264085 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-VolumesNegativeTest-559576417 Jul 11 14:47:34.264085 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: ab6df275ee124e1a90efacdf16c6fa2e Jul 11 14:47:34.264085 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-VolumesNegativeTest-559576417 Jul 11 14:47:34.264085 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-VolumesNegativeTest-559576417-project Jul 11 14:47:34.264085 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:47:34.264085 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:47:34.264085 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 6352e8010a3d4163a5b9c3fd6d68b37b Jul 11 14:47:34.264085 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-VolumesNegativeTest-559576417-project Jul 11 14:47:34.264085 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Jul 11 14:47:34.264085 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-223518008", "metadata": {"Type": "work"}}} Jul 11 14:47:34.264085 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: Jul 11 14:47:34.264085 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-223518008', 'metadata': {'Type': 'work'}}}} Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 279, in validate Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise error Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi On instance['volume']: Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi '' Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:34.265547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 308, in validate Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Jul 11 14:47:34.268673 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Jul 11 14:47:34.277773 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Jul 11 14:47:34.277773 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.277773 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-55371c43-8bc8-499a-85e4-a85cb1c607b4 tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:34.277773 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-55371c43-8bc8-499a-85e4-a85cb1c607b4 tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] 10.222.0.72 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.015806 Jul 11 14:47:34.277773 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 494/956] 10.222.0.72 () {60 vars in 1155 bytes} [Mon Jul 11 14:47:34 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 18 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Jul 11 14:47:34.278416 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e01dec05-20e3-43f2-b305-91a040268b9b tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] Calling method '.version_select at 0x7f592b7c84c0>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:34.307595 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-e01dec05-20e3-43f2-b305-91a040268b9b tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] HTTP exception thrown: Volume tempest-invalid-1843985792 could not be found. Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e01dec05-20e3-43f2-b305-91a040268b9b tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] Request method failure captured: Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1843985792 HTTP/1.1 Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDgFLT1oXLsBaNjknWAIzrQn0Mli7JeYXx0phdGxPoDsISV14y9csZluXDN3sCK6CL0i_6R8A48YD2G7_DL-vrbqx7Dg9tSLxnql8JraNNNdFUKmjN0Xf66l9kBStt9vw8LAG3JFwWkCHWkTzH2xqY7hFKuI4LrrFtj4LHTj-iR-1fk Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: ab6df275ee124e1a90efacdf16c6fa2e Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-VolumesNegativeTest-559576417 Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:47:34.310970 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:47:34.312990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/ab6df275ee124e1a90efacdf16c6fa2e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/ab6df275ee124e1a90efacdf16c6fa2e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_ab6df275ee124e1a90efacdf16c6fa2e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/ab6df275ee124e1a90efacdf16c6fa2e"}]}] Jul 11 14:47:34.312990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-VolumesNegativeTest-559576417 Jul 11 14:47:34.312990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: ab6df275ee124e1a90efacdf16c6fa2e Jul 11 14:47:34.312990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-VolumesNegativeTest-559576417 Jul 11 14:47:34.312990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-VolumesNegativeTest-559576417-project Jul 11 14:47:34.312990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:47:34.312990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:47:34.312990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 6352e8010a3d4163a5b9c3fd6d68b37b Jul 11 14:47:34.312990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-VolumesNegativeTest-559576417-project Jul 11 14:47:34.312990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: Jul 11 14:47:34.312990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Volume tempest-invalid-1843985792 could not be found. Jul 11 14:47:34.312990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': 'tempest-invalid-1843985792'} Jul 11 14:47:34.312990 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1843985792 could not be found. (HTTP 404) (Request-ID: req-674392c2-ad31-4e5c-abd8-b25b9fe3ec07) Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:34.313942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 11 14:47:34.315268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1843985792 could not be found. Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:47:34.316675 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1843985792 could not be found. Jul 11 14:47:34.318135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.318135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:34.318135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.318135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.318135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:34.318135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:34.318135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:47:34.318135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:47:34.318135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1843985792 could not be found. Jul 11 14:47:34.318135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.318135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e01dec05-20e3-43f2-b305-91a040268b9b tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] Returning 404 to user: Volume tempest-invalid-1843985792 could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:34.318135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e01dec05-20e3-43f2-b305-91a040268b9b tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] 10.222.0.72 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1843985792" status: 404 len: 99 microversion: 2.1 time: 0.041682 Jul 11 14:47:34.318135 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 463/957] 10.222.0.72 () {58 vars in 1217 bytes} [Mon Jul 11 14:47:34 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1843985792 => generated 99 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Jul 11 14:47:34.328487 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e60cce90-808f-4811-a5c6-03aa5c520944 tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] 10.222.0.72 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.007023 Jul 11 14:47:34.330017 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 495/958] 10.222.0.72 () {58 vars in 1139 bytes} [Mon Jul 11 14:47:34 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 9 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Jul 11 14:47:34.346270 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-45a2306d-d579-40f1-9d21-40f2562e6143 tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] 10.222.0.72 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.006931 Jul 11 14:47:34.347269 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 464/959] 10.222.0.72 () {58 vars in 1136 bytes} [Mon Jul 11 14:47:34 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 10 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Jul 11 14:47:34.359010 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-390adcb3-109b-4601-8d68-d75aa2a472d4 tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] Calling method '.version_select at 0x7f59302acc10>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:34.390380 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-390adcb3-109b-4601-8d68-d75aa2a472d4 tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] HTTP exception thrown: Volume 18fbf678-0254-46a2-8aca-36e1500cfc16 could not be found. Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-390adcb3-109b-4601-8d68-d75aa2a472d4 tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] Request method failure captured: Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: DELETE /compute/v2.1/os-volumes/18fbf678-0254-46a2-8aca-36e1500cfc16 HTTP/1.1 Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDgFLT1oXLsBaNjknWAIzrQn0Mli7JeYXx0phdGxPoDsISV14y9csZluXDN3sCK6CL0i_6R8A48YD2G7_DL-vrbqx7Dg9tSLxnql8JraNNNdFUKmjN0Xf66l9kBStt9vw8LAG3JFwWkCHWkTzH2xqY7hFKuI4LrrFtj4LHTj-iR-1fk Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: ab6df275ee124e1a90efacdf16c6fa2e Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-VolumesNegativeTest-559576417 Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:47:34.393133 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:47:34.395811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/ab6df275ee124e1a90efacdf16c6fa2e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/ab6df275ee124e1a90efacdf16c6fa2e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_ab6df275ee124e1a90efacdf16c6fa2e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/ab6df275ee124e1a90efacdf16c6fa2e"}]}] Jul 11 14:47:34.395811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-VolumesNegativeTest-559576417 Jul 11 14:47:34.395811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: ab6df275ee124e1a90efacdf16c6fa2e Jul 11 14:47:34.395811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-VolumesNegativeTest-559576417 Jul 11 14:47:34.395811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-VolumesNegativeTest-559576417-project Jul 11 14:47:34.395811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:47:34.395811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:47:34.395811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 6352e8010a3d4163a5b9c3fd6d68b37b Jul 11 14:47:34.395811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-VolumesNegativeTest-559576417-project Jul 11 14:47:34.395811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: Jul 11 14:47:34.395811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Volume 18fbf678-0254-46a2-8aca-36e1500cfc16 could not be found. Jul 11 14:47:34.395811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '18fbf678-0254-46a2-8aca-36e1500cfc16'} Jul 11 14:47:34.395811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 18fbf678-0254-46a2-8aca-36e1500cfc16 could not be found. (HTTP 404) (Request-ID: req-32602186-8cfe-439f-af64-f65b69ec89a2) Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:34.397514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._delete(loc) Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Jul 11 14:47:34.400105 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 18fbf678-0254-46a2-8aca-36e1500cfc16 could not be found. Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Jul 11 14:47:34.402847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:47:34.405675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 18fbf678-0254-46a2-8aca-36e1500cfc16 could not be found. Jul 11 14:47:34.405675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.405675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:34.405675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.405675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.405675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:34.405675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:34.405675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:47:34.405675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:47:34.405675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 18fbf678-0254-46a2-8aca-36e1500cfc16 could not be found. Jul 11 14:47:34.405675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.405675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-390adcb3-109b-4601-8d68-d75aa2a472d4 tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] Returning 404 to user: Volume 18fbf678-0254-46a2-8aca-36e1500cfc16 could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:34.405675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-390adcb3-109b-4601-8d68-d75aa2a472d4 tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] 10.222.0.72 "DELETE /compute/v2.1/os-volumes/18fbf678-0254-46a2-8aca-36e1500cfc16" status: 404 len: 109 microversion: 2.1 time: 0.040248 Jul 11 14:47:34.405675 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 496/960] 10.222.0.72 () {58 vars in 1247 bytes} [Mon Jul 11 14:47:34 2022] DELETE /compute/v2.1/os-volumes/18fbf678-0254-46a2-8aca-36e1500cfc16 => generated 109 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:47:34.410484 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-0f95c2b0-d2d7-46ef-a512-dcab8fec43ea tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] Calling method '.version_select at 0x7f592b7c8d30>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:34.440359 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-0f95c2b0-d2d7-46ef-a512-dcab8fec43ea tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] HTTP exception thrown: Volume ee4357cc-cc1b-47da-a3bb-0a7a156b5f23 could not be found. Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-0f95c2b0-d2d7-46ef-a512-dcab8fec43ea tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] Request method failure captured: Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/os-volumes/ee4357cc-cc1b-47da-a3bb-0a7a156b5f23 HTTP/1.1 Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDgFLT1oXLsBaNjknWAIzrQn0Mli7JeYXx0phdGxPoDsISV14y9csZluXDN3sCK6CL0i_6R8A48YD2G7_DL-vrbqx7Dg9tSLxnql8JraNNNdFUKmjN0Xf66l9kBStt9vw8LAG3JFwWkCHWkTzH2xqY7hFKuI4LrrFtj4LHTj-iR-1fk Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: ab6df275ee124e1a90efacdf16c6fa2e Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-VolumesNegativeTest-559576417 Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:47:34.442602 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:47:34.446000 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/ab6df275ee124e1a90efacdf16c6fa2e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/ab6df275ee124e1a90efacdf16c6fa2e"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_ab6df275ee124e1a90efacdf16c6fa2e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/ab6df275ee124e1a90efacdf16c6fa2e"}]}] Jul 11 14:47:34.446000 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-VolumesNegativeTest-559576417 Jul 11 14:47:34.446000 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: ab6df275ee124e1a90efacdf16c6fa2e Jul 11 14:47:34.446000 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-VolumesNegativeTest-559576417 Jul 11 14:47:34.446000 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-VolumesNegativeTest-559576417-project Jul 11 14:47:34.446000 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:47:34.446000 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:47:34.446000 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 6352e8010a3d4163a5b9c3fd6d68b37b Jul 11 14:47:34.446000 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-VolumesNegativeTest-559576417-project Jul 11 14:47:34.446000 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: Jul 11 14:47:34.446000 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Volume ee4357cc-cc1b-47da-a3bb-0a7a156b5f23 could not be found. Jul 11 14:47:34.446000 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': 'ee4357cc-cc1b-47da-a3bb-0a7a156b5f23'} Jul 11 14:47:34.446000 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume ee4357cc-cc1b-47da-a3bb-0a7a156b5f23 could not be found. (HTTP 404) (Request-ID: req-50597364-514e-412f-af35-299f542c54e8) Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:34.447698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 11 14:47:34.450141 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume ee4357cc-cc1b-47da-a3bb-0a7a156b5f23 could not be found. Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 11 14:47:34.452316 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:47:34.455138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume ee4357cc-cc1b-47da-a3bb-0a7a156b5f23 could not be found. Jul 11 14:47:34.455138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.455138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:47:34.455138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.455138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:47:34.455138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:47:34.455138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:47:34.455138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:47:34.455138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:47:34.455138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume ee4357cc-cc1b-47da-a3bb-0a7a156b5f23 could not be found. Jul 11 14:47:34.455138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:47:34.455138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-0f95c2b0-d2d7-46ef-a512-dcab8fec43ea tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] Returning 404 to user: Volume ee4357cc-cc1b-47da-a3bb-0a7a156b5f23 could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:47:34.455138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-0f95c2b0-d2d7-46ef-a512-dcab8fec43ea tempest-VolumesNegativeTest-559576417 tempest-VolumesNegativeTest-559576417-project] 10.222.0.72 "GET /compute/v2.1/os-volumes/ee4357cc-cc1b-47da-a3bb-0a7a156b5f23" status: 404 len: 109 microversion: 2.1 time: 0.039675 Jul 11 14:47:34.455138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 465/961] 10.222.0.72 () {58 vars in 1244 bytes} [Mon Jul 11 14:47:34 2022] GET /compute/v2.1/os-volumes/ee4357cc-cc1b-47da-a3bb-0a7a156b5f23 => generated 109 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:47:44.419993 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-780e1f61-af0e-484c-8772-ad57302bb893 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1194932779", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-2098849656", "security_groups": [{"name": "tempest-secgroup-smoke-2075789476"}], "block_device_mapping_v2": [{"uuid": "b29153e8-7fc8-4c30-aa1f-166dd95bdb16", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "e7191ae9-3a75-447f-ae05-2827e78eae91"}]}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:47:44.442233 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=5 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:47:44.443873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=10 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:47:44.692394 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.network.neutron [None req-780e1f61-af0e-484c-8772-ad57302bb893 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] validate_networks() for [('e7191ae9-3a75-447f-ae05-2827e78eae91', None, None, None, None, None)] {{(pid=130159) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:47:45.210059 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-780e1f61-af0e-484c-8772-ad57302bb893 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Flavor limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:47:45.211106 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-780e1f61-af0e-484c-8772-ad57302bb893 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Image limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:47:45.212098 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-780e1f61-af0e-484c-8772-ad57302bb893 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Flavor pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:47:45.213022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-780e1f61-af0e-484c-8772-ad57302bb893 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Image pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:47:45.213924 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-780e1f61-af0e-484c-8772-ad57302bb893 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:47:45.228273 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-780e1f61-af0e-484c-8772-ad57302bb893 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:45.228717 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-780e1f61-af0e-484c-8772-ad57302bb893 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:45.229533 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-780e1f61-af0e-484c-8772-ad57302bb893 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:45.230849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-780e1f61-af0e-484c-8772-ad57302bb893 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:45.251570 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-780e1f61-af0e-484c-8772-ad57302bb893 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Getting quotas for project 3d541b7a72dc4b4494a42bda6fa75026. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:47:45.256719 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-780e1f61-af0e-484c-8772-ad57302bb893 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Getting quotas for user 0f8e300cd96e446db7430044e69d997a and project 3d541b7a72dc4b4494a42bda6fa75026. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:47:45.359161 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-780e1f61-af0e-484c-8772-ad57302bb893 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Going to run 1 instances... {{(pid=130159) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:47:45.363015 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-780e1f61-af0e-484c-8772-ad57302bb893 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='b29153e8-7fc8-4c30-aa1f-166dd95bdb16',volume_size=None,volume_type=None)] {{(pid=130159) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:47:45.553899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-780e1f61-af0e-484c-8772-ad57302bb893 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.140705 Jul 11 14:47:45.563403 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 497/962] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:47:44 2022] POST /compute/v2.1/servers => generated 406 bytes in 1144 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 11 14:47:45.578053 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=44,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:47:45.579147 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=8 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:47:45.588302 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-722e8eba-0518-41aa-8d7d-739b192628a1 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:45.588930 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-722e8eba-0518-41aa-8d7d-739b192628a1 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:45.669882 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-722e8eba-0518-41aa-8d7d-739b192628a1 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:45.679302 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-722e8eba-0518-41aa-8d7d-739b192628a1 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1095 microversion: 2.1 time: 0.113696 Jul 11 14:47:45.682226 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 466/963] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:45 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1095 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:47:46.702702 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-b1af2536-f7cf-4a94-a0f7-f6d92c277c02 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:46.703473 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-b1af2536-f7cf-4a94-a0f7-f6d92c277c02 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:46.712524 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b1af2536-f7cf-4a94-a0f7-f6d92c277c02 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:46.713163 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b1af2536-f7cf-4a94-a0f7-f6d92c277c02 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:46.714096 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b1af2536-f7cf-4a94-a0f7-f6d92c277c02 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:46.714888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b1af2536-f7cf-4a94-a0f7-f6d92c277c02 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:46.802154 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-b1af2536-f7cf-4a94-a0f7-f6d92c277c02 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:46.816929 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-b1af2536-f7cf-4a94-a0f7-f6d92c277c02 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1193 microversion: 2.1 time: 0.120335 Jul 11 14:47:46.826130 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 498/964] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:46 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1193 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:47:47.840444 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d510a0da-9735-49f1-8817-4610d4bfa34d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:47.841646 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d510a0da-9735-49f1-8817-4610d4bfa34d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:47.853282 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d510a0da-9735-49f1-8817-4610d4bfa34d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:47.854059 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d510a0da-9735-49f1-8817-4610d4bfa34d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:47.855451 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d510a0da-9735-49f1-8817-4610d4bfa34d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:47.856195 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d510a0da-9735-49f1-8817-4610d4bfa34d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:47.959965 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d510a0da-9735-49f1-8817-4610d4bfa34d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:47.969488 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d510a0da-9735-49f1-8817-4610d4bfa34d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1193 microversion: 2.1 time: 0.135638 Jul 11 14:47:47.971065 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 467/965] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:47 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1193 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:47:48.995378 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-425f3938-23e7-4b0f-b5e8-5a0fad36e70e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:48.995976 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-425f3938-23e7-4b0f-b5e8-5a0fad36e70e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:49.003722 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-425f3938-23e7-4b0f-b5e8-5a0fad36e70e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:49.004241 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-425f3938-23e7-4b0f-b5e8-5a0fad36e70e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:49.005165 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-425f3938-23e7-4b0f-b5e8-5a0fad36e70e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:49.005656 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-425f3938-23e7-4b0f-b5e8-5a0fad36e70e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:49.077877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-425f3938-23e7-4b0f-b5e8-5a0fad36e70e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:49.092854 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-425f3938-23e7-4b0f-b5e8-5a0fad36e70e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1193 microversion: 2.1 time: 0.101495 Jul 11 14:47:49.094028 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 499/966] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:48 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1193 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:47:50.122946 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d6e545e2-aa26-4b4b-b982-f6109d2e29a0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:50.124987 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d6e545e2-aa26-4b4b-b982-f6109d2e29a0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:50.138042 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d6e545e2-aa26-4b4b-b982-f6109d2e29a0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:50.138819 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d6e545e2-aa26-4b4b-b982-f6109d2e29a0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:50.140227 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d6e545e2-aa26-4b4b-b982-f6109d2e29a0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:50.140972 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d6e545e2-aa26-4b4b-b982-f6109d2e29a0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:50.289279 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d6e545e2-aa26-4b4b-b982-f6109d2e29a0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:50.305674 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d6e545e2-aa26-4b4b-b982-f6109d2e29a0 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1211 microversion: 2.1 time: 0.190150 Jul 11 14:47:50.306686 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 468/967] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:50 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1211 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:47:51.327879 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1732bcd5-2621-45b5-8e01-326709cc6edd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:51.328796 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1732bcd5-2621-45b5-8e01-326709cc6edd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:51.345301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1732bcd5-2621-45b5-8e01-326709cc6edd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:51.345757 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1732bcd5-2621-45b5-8e01-326709cc6edd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:51.346595 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1732bcd5-2621-45b5-8e01-326709cc6edd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:51.347030 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1732bcd5-2621-45b5-8e01-326709cc6edd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:51.508020 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1732bcd5-2621-45b5-8e01-326709cc6edd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:51.522222 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1732bcd5-2621-45b5-8e01-326709cc6edd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1279 microversion: 2.1 time: 0.199257 Jul 11 14:47:51.523321 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 500/968] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:51 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1279 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:47:52.547243 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-75eae341-b0c9-40f0-baaf-24b508e8dd29 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:52.548080 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-75eae341-b0c9-40f0-baaf-24b508e8dd29 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:52.555428 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-75eae341-b0c9-40f0-baaf-24b508e8dd29 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:52.555882 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-75eae341-b0c9-40f0-baaf-24b508e8dd29 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:52.556686 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-75eae341-b0c9-40f0-baaf-24b508e8dd29 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:52.557133 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-75eae341-b0c9-40f0-baaf-24b508e8dd29 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:52.826200 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-75eae341-b0c9-40f0-baaf-24b508e8dd29 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:52.843109 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-75eae341-b0c9-40f0-baaf-24b508e8dd29 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1279 microversion: 2.1 time: 0.299496 Jul 11 14:47:52.844249 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 469/969] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:52 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1279 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:47:52.916239 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-713d7cbc-aefa-4feb-91bd-358dde648f28 req-6d3de7ba-1231-4af3-9261-48ed0f2c9db4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e2f87e01-f360-4b6f-afde-6a64be9f72f7", "tag": "e9f1af2a-fec1-4461-bf84-aca44600c87d"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:47:52.929681 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-713d7cbc-aefa-4feb-91bd-358dde648f28 req-6d3de7ba-1231-4af3-9261-48ed0f2c9db4 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:52.930924 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-713d7cbc-aefa-4feb-91bd-358dde648f28 req-6d3de7ba-1231-4af3-9261-48ed0f2c9db4 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:52.945106 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-713d7cbc-aefa-4feb-91bd-358dde648f28 req-6d3de7ba-1231-4af3-9261-48ed0f2c9db4 service nova] Creating event network-changed:e9f1af2a-fec1-4461-bf84-aca44600c87d for instance e2f87e01-f360-4b6f-afde-6a64be9f72f7 on hv-cinder-maste Jul 11 14:47:52.950962 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-713d7cbc-aefa-4feb-91bd-358dde648f28 req-6d3de7ba-1231-4af3-9261-48ed0f2c9db4 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042250 Jul 11 14:47:52.951587 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 501/970] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:47:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:47:53.864541 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-e2d80813-c4b0-4d64-8182-bb601f4e3c29 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:53.865164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-e2d80813-c4b0-4d64-8182-bb601f4e3c29 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:53.872194 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e2d80813-c4b0-4d64-8182-bb601f4e3c29 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:53.872680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e2d80813-c4b0-4d64-8182-bb601f4e3c29 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:53.873436 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e2d80813-c4b0-4d64-8182-bb601f4e3c29 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:53.873875 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-e2d80813-c4b0-4d64-8182-bb601f4e3c29 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:54.029334 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-e2d80813-c4b0-4d64-8182-bb601f4e3c29 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:54.038332 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-e2d80813-c4b0-4d64-8182-bb601f4e3c29 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1279 microversion: 2.1 time: 0.178408 Jul 11 14:47:54.038987 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 470/971] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:53 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1279 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:47:55.057991 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-0fb2b5b7-a28e-4a6c-a335-80d7ca637266 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:55.058763 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-0fb2b5b7-a28e-4a6c-a335-80d7ca637266 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:55.065702 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0fb2b5b7-a28e-4a6c-a335-80d7ca637266 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:55.066298 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0fb2b5b7-a28e-4a6c-a335-80d7ca637266 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:55.067257 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0fb2b5b7-a28e-4a6c-a335-80d7ca637266 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:55.067793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0fb2b5b7-a28e-4a6c-a335-80d7ca637266 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:55.222995 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0fb2b5b7-a28e-4a6c-a335-80d7ca637266 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:55.233899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-0fb2b5b7-a28e-4a6c-a335-80d7ca637266 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1279 microversion: 2.1 time: 0.180997 Jul 11 14:47:55.234535 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 502/972] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:55 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1279 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:47:56.260708 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9348d437-96f2-4207-b6fe-1da8f036e5ca tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:56.261981 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-9348d437-96f2-4207-b6fe-1da8f036e5ca tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:56.266259 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=2 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:47:56.274344 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9348d437-96f2-4207-b6fe-1da8f036e5ca tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:56.274829 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9348d437-96f2-4207-b6fe-1da8f036e5ca tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:56.275826 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9348d437-96f2-4207-b6fe-1da8f036e5ca tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:56.276879 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9348d437-96f2-4207-b6fe-1da8f036e5ca tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:56.710151 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-9348d437-96f2-4207-b6fe-1da8f036e5ca tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:56.741049 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9348d437-96f2-4207-b6fe-1da8f036e5ca tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1279 microversion: 2.1 time: 0.486982 Jul 11 14:47:56.741973 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 471/973] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:56 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1279 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:47:57.768749 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-6f4c76cb-89c0-4468-893d-ebd18de5dd80 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:57.769374 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-6f4c76cb-89c0-4468-893d-ebd18de5dd80 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:57.771501 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=2 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:47:57.772243 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=6 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:47:57.785280 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6f4c76cb-89c0-4468-893d-ebd18de5dd80 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:57.786082 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6f4c76cb-89c0-4468-893d-ebd18de5dd80 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:57.787541 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6f4c76cb-89c0-4468-893d-ebd18de5dd80 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:57.788315 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6f4c76cb-89c0-4468-893d-ebd18de5dd80 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:57.971252 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-6f4c76cb-89c0-4468-893d-ebd18de5dd80 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:57.980892 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-6f4c76cb-89c0-4468-893d-ebd18de5dd80 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1279 microversion: 2.1 time: 0.216395 Jul 11 14:47:57.981490 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 503/974] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:57 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1279 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:47:58.999431 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-4cf93ba8-59c3-4a33-a30b-220c72279863 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:47:59.000299 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-4cf93ba8-59c3-4a33-a30b-220c72279863 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:47:59.007698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4cf93ba8-59c3-4a33-a30b-220c72279863 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:59.008210 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4cf93ba8-59c3-4a33-a30b-220c72279863 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:59.009052 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4cf93ba8-59c3-4a33-a30b-220c72279863 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:47:59.009548 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4cf93ba8-59c3-4a33-a30b-220c72279863 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:47:59.158150 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4cf93ba8-59c3-4a33-a30b-220c72279863 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:47:59.166679 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-4cf93ba8-59c3-4a33-a30b-220c72279863 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1279 microversion: 2.1 time: 0.171389 Jul 11 14:47:59.167349 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 472/975] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:47:58 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1279 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:48:00.184530 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d01a185a-b14d-4d00-bdee-920e2b6a891c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:00.185136 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d01a185a-b14d-4d00-bdee-920e2b6a891c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:00.192467 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d01a185a-b14d-4d00-bdee-920e2b6a891c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:00.192915 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d01a185a-b14d-4d00-bdee-920e2b6a891c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:00.193911 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d01a185a-b14d-4d00-bdee-920e2b6a891c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:00.194385 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d01a185a-b14d-4d00-bdee-920e2b6a891c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:00.345735 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d01a185a-b14d-4d00-bdee-920e2b6a891c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:00.354031 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-d01a185a-b14d-4d00-bdee-920e2b6a891c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1279 microversion: 2.1 time: 0.173603 Jul 11 14:48:00.354738 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 504/976] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:00 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1279 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:01.372481 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-abb92df6-0c71-45e8-ba38-a2f3df6e0dcc tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:01.373222 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-abb92df6-0c71-45e8-ba38-a2f3df6e0dcc tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:01.380625 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-abb92df6-0c71-45e8-ba38-a2f3df6e0dcc tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:01.381073 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-abb92df6-0c71-45e8-ba38-a2f3df6e0dcc tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:01.381857 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-abb92df6-0c71-45e8-ba38-a2f3df6e0dcc tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:01.382294 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-abb92df6-0c71-45e8-ba38-a2f3df6e0dcc tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:01.534349 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-abb92df6-0c71-45e8-ba38-a2f3df6e0dcc tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:01.543611 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-abb92df6-0c71-45e8-ba38-a2f3df6e0dcc tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1279 microversion: 2.1 time: 0.175225 Jul 11 14:48:01.544219 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 473/977] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:01 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1279 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:48:02.561986 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d5f2c1d7-2451-4e6a-ad02-b38a8971ab5d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:02.562642 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d5f2c1d7-2451-4e6a-ad02-b38a8971ab5d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:02.569750 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d5f2c1d7-2451-4e6a-ad02-b38a8971ab5d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:02.570206 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d5f2c1d7-2451-4e6a-ad02-b38a8971ab5d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:02.571037 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d5f2c1d7-2451-4e6a-ad02-b38a8971ab5d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:02.571484 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d5f2c1d7-2451-4e6a-ad02-b38a8971ab5d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:02.744121 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d5f2c1d7-2451-4e6a-ad02-b38a8971ab5d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:02.753530 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-d5f2c1d7-2451-4e6a-ad02-b38a8971ab5d tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1279 microversion: 2.1 time: 0.195759 Jul 11 14:48:02.754178 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 505/978] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:02 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1279 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:03.781262 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d069b980-e5ad-488f-a812-1446c5b27e8c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:03.781940 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d069b980-e5ad-488f-a812-1446c5b27e8c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:03.789490 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d069b980-e5ad-488f-a812-1446c5b27e8c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:03.789945 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d069b980-e5ad-488f-a812-1446c5b27e8c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:03.790809 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d069b980-e5ad-488f-a812-1446c5b27e8c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:03.791261 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d069b980-e5ad-488f-a812-1446c5b27e8c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:03.981654 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d069b980-e5ad-488f-a812-1446c5b27e8c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:03.990397 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d069b980-e5ad-488f-a812-1446c5b27e8c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1279 microversion: 2.1 time: 0.213117 Jul 11 14:48:03.991023 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 474/979] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:03 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1279 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:48:05.008077 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-ab1e893b-392d-47b7-9bd3-159efeeb5859 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:05.008707 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-ab1e893b-392d-47b7-9bd3-159efeeb5859 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:05.016002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ab1e893b-392d-47b7-9bd3-159efeeb5859 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:05.017170 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ab1e893b-392d-47b7-9bd3-159efeeb5859 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:05.018192 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ab1e893b-392d-47b7-9bd3-159efeeb5859 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:05.018894 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-ab1e893b-392d-47b7-9bd3-159efeeb5859 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:05.214514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-ab1e893b-392d-47b7-9bd3-159efeeb5859 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:05.224472 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-ab1e893b-392d-47b7-9bd3-159efeeb5859 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1279 microversion: 2.1 time: 0.220191 Jul 11 14:48:05.225394 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 506/980] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:05 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1279 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:06.244291 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-95065f19-3b55-446e-806e-3c0113dbfdba tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:06.245324 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-95065f19-3b55-446e-806e-3c0113dbfdba tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:06.253824 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-95065f19-3b55-446e-806e-3c0113dbfdba tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:06.254723 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-95065f19-3b55-446e-806e-3c0113dbfdba tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:06.255712 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-95065f19-3b55-446e-806e-3c0113dbfdba tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:06.256811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-95065f19-3b55-446e-806e-3c0113dbfdba tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:06.447376 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-95065f19-3b55-446e-806e-3c0113dbfdba tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:06.467133 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-95065f19-3b55-446e-806e-3c0113dbfdba tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1279 microversion: 2.1 time: 0.228021 Jul 11 14:48:06.468145 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 475/981] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:06 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1279 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:07.488485 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-0ce3568a-a7c8-458c-8f16-458105a2582f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:07.489263 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-0ce3568a-a7c8-458c-8f16-458105a2582f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:07.496306 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0ce3568a-a7c8-458c-8f16-458105a2582f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:07.496755 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0ce3568a-a7c8-458c-8f16-458105a2582f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:07.497544 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0ce3568a-a7c8-458c-8f16-458105a2582f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:07.497984 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0ce3568a-a7c8-458c-8f16-458105a2582f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:07.666422 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0ce3568a-a7c8-458c-8f16-458105a2582f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:07.678274 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-0ce3568a-a7c8-458c-8f16-458105a2582f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1279 microversion: 2.1 time: 0.194228 Jul 11 14:48:07.679714 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 507/982] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:07 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1279 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:08.697522 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-95fad9e6-c948-4a04-b05c-4f120fed7f10 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:08.698134 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-95fad9e6-c948-4a04-b05c-4f120fed7f10 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:08.705322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-95fad9e6-c948-4a04-b05c-4f120fed7f10 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:08.705685 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-95fad9e6-c948-4a04-b05c-4f120fed7f10 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:08.706497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-95fad9e6-c948-4a04-b05c-4f120fed7f10 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:08.706939 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-95fad9e6-c948-4a04-b05c-4f120fed7f10 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:08.900618 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-95fad9e6-c948-4a04-b05c-4f120fed7f10 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:08.910928 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-95fad9e6-c948-4a04-b05c-4f120fed7f10 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1279 microversion: 2.1 time: 0.217053 Jul 11 14:48:08.911700 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 476/983] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:08 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1279 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:09.929745 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e15b3df7-e397-4e9b-baff-587912c51432 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:09.930732 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e15b3df7-e397-4e9b-baff-587912c51432 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:09.941602 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e15b3df7-e397-4e9b-baff-587912c51432 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:09.942668 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e15b3df7-e397-4e9b-baff-587912c51432 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:09.943635 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e15b3df7-e397-4e9b-baff-587912c51432 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:09.944398 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e15b3df7-e397-4e9b-baff-587912c51432 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:10.118264 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e15b3df7-e397-4e9b-baff-587912c51432 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:10.126990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e15b3df7-e397-4e9b-baff-587912c51432 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1279 microversion: 2.1 time: 0.201437 Jul 11 14:48:10.127681 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 508/984] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:09 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1279 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:48:11.144225 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3c1908ff-6ee5-428e-8f49-86e80ef19d84 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:11.145153 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-3c1908ff-6ee5-428e-8f49-86e80ef19d84 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:11.151909 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3c1908ff-6ee5-428e-8f49-86e80ef19d84 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:11.152417 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3c1908ff-6ee5-428e-8f49-86e80ef19d84 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:11.153288 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3c1908ff-6ee5-428e-8f49-86e80ef19d84 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:11.153782 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3c1908ff-6ee5-428e-8f49-86e80ef19d84 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:11.298159 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-3c1908ff-6ee5-428e-8f49-86e80ef19d84 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:11.312345 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-3c1908ff-6ee5-428e-8f49-86e80ef19d84 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1267 microversion: 2.1 time: 0.171759 Jul 11 14:48:11.314005 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 477/985] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:11 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1267 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:48:12.317336 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-32320e0a-0bb9-4f12-a135-4d0d9fdb2441 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:12.318565 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-32320e0a-0bb9-4f12-a135-4d0d9fdb2441 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:12.325708 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-32320e0a-0bb9-4f12-a135-4d0d9fdb2441 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:12.326268 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-32320e0a-0bb9-4f12-a135-4d0d9fdb2441 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:12.327987 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-32320e0a-0bb9-4f12-a135-4d0d9fdb2441 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:12.328705 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-32320e0a-0bb9-4f12-a135-4d0d9fdb2441 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:12.334375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-194ccd4f-75cc-4179-87fb-9d717eaea109 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:12.335111 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-194ccd4f-75cc-4179-87fb-9d717eaea109 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:12.343656 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-194ccd4f-75cc-4179-87fb-9d717eaea109 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:12.344643 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-194ccd4f-75cc-4179-87fb-9d717eaea109 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:12.346389 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-194ccd4f-75cc-4179-87fb-9d717eaea109 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:12.346967 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-194ccd4f-75cc-4179-87fb-9d717eaea109 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:12.368242 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-32320e0a-0bb9-4f12-a135-4d0d9fdb2441 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.054829 Jul 11 14:48:12.368855 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 509/986] 10.222.0.72 () {60 vars in 1342 bytes} [Mon Jul 11 14:48:12 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad/os-volume_attachments => generated 25 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:12.453443 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e3d83f79-b0d0-4779-97f5-d547c0467347 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:12.455080 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e3d83f79-b0d0-4779-97f5-d547c0467347 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:12.463445 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e3d83f79-b0d0-4779-97f5-d547c0467347 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:12.464281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e3d83f79-b0d0-4779-97f5-d547c0467347 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:12.465924 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e3d83f79-b0d0-4779-97f5-d547c0467347 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:12.466712 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e3d83f79-b0d0-4779-97f5-d547c0467347 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:12.511227 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e3d83f79-b0d0-4779-97f5-d547c0467347 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Going to try to terminate instance {{(pid=130159) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:48:12.581687 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-194ccd4f-75cc-4179-87fb-9d717eaea109 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:12.597960 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-194ccd4f-75cc-4179-87fb-9d717eaea109 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1267 microversion: 2.1 time: 0.269206 Jul 11 14:48:12.599186 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 478/987] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:12 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1267 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:48:12.646354 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e3d83f79-b0d0-4779-97f5-d547c0467347 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "DELETE /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 204 len: 0 microversion: 2.20 time: 0.197625 Jul 11 14:48:12.647608 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 510/988] 10.222.0.72 () {60 vars in 1279 bytes} [Mon Jul 11 14:48:12 2022] DELETE /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 0 bytes in 201 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 11 14:48:12.663889 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8e2c2641-aa41-4652-8bc1-b9ac3cfb5d31 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:12.665086 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-8e2c2641-aa41-4652-8bc1-b9ac3cfb5d31 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:12.676618 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8e2c2641-aa41-4652-8bc1-b9ac3cfb5d31 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:12.677750 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8e2c2641-aa41-4652-8bc1-b9ac3cfb5d31 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:12.679536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8e2c2641-aa41-4652-8bc1-b9ac3cfb5d31 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:12.680386 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8e2c2641-aa41-4652-8bc1-b9ac3cfb5d31 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:12.852047 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8e2c2641-aa41-4652-8bc1-b9ac3cfb5d31 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:12.866305 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8e2c2641-aa41-4652-8bc1-b9ac3cfb5d31 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:12.868559 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8e2c2641-aa41-4652-8bc1-b9ac3cfb5d31 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:12.871049 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8e2c2641-aa41-4652-8bc1-b9ac3cfb5d31 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1722 microversion: 2.20 time: 0.216126 Jul 11 14:48:12.871415 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 479/989] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:48:12 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1722 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:48:13.620022 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2f093e34-6055-43d0-9621-453d137239e6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:13.621207 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2f093e34-6055-43d0-9621-453d137239e6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:13.632255 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2f093e34-6055-43d0-9621-453d137239e6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:13.633037 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2f093e34-6055-43d0-9621-453d137239e6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:13.634457 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2f093e34-6055-43d0-9621-453d137239e6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:13.635083 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2f093e34-6055-43d0-9621-453d137239e6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:13.811800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-2f093e34-6055-43d0-9621-453d137239e6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:13.821401 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2f093e34-6055-43d0-9621-453d137239e6 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1267 microversion: 2.1 time: 0.207641 Jul 11 14:48:13.821989 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 511/990] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:13 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1267 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:13.899222 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2362f8a6-8552-4a85-b99f-f2c4a2146591 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:13.900045 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2362f8a6-8552-4a85-b99f-f2c4a2146591 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:13.908296 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2362f8a6-8552-4a85-b99f-f2c4a2146591 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:13.908772 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2362f8a6-8552-4a85-b99f-f2c4a2146591 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:13.909889 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2362f8a6-8552-4a85-b99f-f2c4a2146591 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:13.910335 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2362f8a6-8552-4a85-b99f-f2c4a2146591 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:14.091181 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2362f8a6-8552-4a85-b99f-f2c4a2146591 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:14.103221 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2362f8a6-8552-4a85-b99f-f2c4a2146591 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:14.104773 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2362f8a6-8552-4a85-b99f-f2c4a2146591 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:14.117237 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2362f8a6-8552-4a85-b99f-f2c4a2146591 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1722 microversion: 2.20 time: 0.222160 Jul 11 14:48:14.118132 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 480/991] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:48:13 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1722 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Jul 11 14:48:14.846531 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-84497057-8130-4498-88f8-5e02e11326c5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:14.848221 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-84497057-8130-4498-88f8-5e02e11326c5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:14.851651 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=49,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:48:14.859724 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84497057-8130-4498-88f8-5e02e11326c5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:14.860293 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84497057-8130-4498-88f8-5e02e11326c5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:14.862221 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84497057-8130-4498-88f8-5e02e11326c5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:14.862857 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84497057-8130-4498-88f8-5e02e11326c5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:14.868884 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:48:15.053408 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-84497057-8130-4498-88f8-5e02e11326c5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:15.061759 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-84497057-8130-4498-88f8-5e02e11326c5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1267 microversion: 2.1 time: 0.220923 Jul 11 14:48:15.062368 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 512/992] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:14 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1267 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:15.141827 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c5be7cb9-906a-41a5-84d6-3e43fdad891b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:15.143008 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c5be7cb9-906a-41a5-84d6-3e43fdad891b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:15.151292 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c5be7cb9-906a-41a5-84d6-3e43fdad891b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:15.151750 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c5be7cb9-906a-41a5-84d6-3e43fdad891b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:15.152544 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c5be7cb9-906a-41a5-84d6-3e43fdad891b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:15.153187 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c5be7cb9-906a-41a5-84d6-3e43fdad891b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:15.228265 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c5be7cb9-906a-41a5-84d6-3e43fdad891b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:15.237333 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c5be7cb9-906a-41a5-84d6-3e43fdad891b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:15.238910 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c5be7cb9-906a-41a5-84d6-3e43fdad891b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': '99a62e49e6d249c2a169ef7122e762ec', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f6606b37accd43619058aef23f10745b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:15.240125 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c5be7cb9-906a-41a5-84d6-3e43fdad891b tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 200 len: 1655 microversion: 2.20 time: 0.102695 Jul 11 14:48:15.240698 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 481/993] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:48:15 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 1655 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Jul 11 14:48:15.352349 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-f374c77c-d2fa-4f3c-9b75-af8ba23d8ea1 req-b9d3ac41-70ec-4638-8f19-05b178f40a86 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "43772fe7-b391-4878-bbea-658602d822ad", "name": "network-vif-deleted", "tag": "35522aa1-427b-4c21-88e8-ce45535459a0"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:48:15.365293 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-f374c77c-d2fa-4f3c-9b75-af8ba23d8ea1 req-b9d3ac41-70ec-4638-8f19-05b178f40a86 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:15.365891 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-f374c77c-d2fa-4f3c-9b75-af8ba23d8ea1 req-b9d3ac41-70ec-4638-8f19-05b178f40a86 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:15.378196 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-f374c77c-d2fa-4f3c-9b75-af8ba23d8ea1 req-b9d3ac41-70ec-4638-8f19-05b178f40a86 service nova] Creating event network-vif-deleted:35522aa1-427b-4c21-88e8-ce45535459a0 for instance 43772fe7-b391-4878-bbea-658602d822ad on hv-cinder-maste Jul 11 14:48:15.384056 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-f374c77c-d2fa-4f3c-9b75-af8ba23d8ea1 req-b9d3ac41-70ec-4638-8f19-05b178f40a86 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036355 Jul 11 14:48:15.387711 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 513/994] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:48:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:48:16.095951 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d60a87a5-b9e1-4d99-8a04-fec0d8ed7bc8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:16.098095 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d60a87a5-b9e1-4d99-8a04-fec0d8ed7bc8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:16.104902 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=35 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:48:16.110204 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d60a87a5-b9e1-4d99-8a04-fec0d8ed7bc8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:16.111593 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d60a87a5-b9e1-4d99-8a04-fec0d8ed7bc8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:16.112628 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d60a87a5-b9e1-4d99-8a04-fec0d8ed7bc8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:16.113267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d60a87a5-b9e1-4d99-8a04-fec0d8ed7bc8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:16.122941 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=91 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:48:16.261199 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-df05b2f4-6579-43f2-a85d-9e6c84c7c226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:16.262753 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-df05b2f4-6579-43f2-a85d-9e6c84c7c226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:16.273043 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-df05b2f4-6579-43f2-a85d-9e6c84c7c226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:16.273858 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-df05b2f4-6579-43f2-a85d-9e6c84c7c226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:16.274745 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-df05b2f4-6579-43f2-a85d-9e6c84c7c226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:16.275902 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-df05b2f4-6579-43f2-a85d-9e6c84c7c226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:16.299064 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-df05b2f4-6579-43f2-a85d-9e6c84c7c226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] HTTP exception thrown: Instance 43772fe7-b391-4878-bbea-658602d822ad could not be found. Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-df05b2f4-6579-43f2-a85d-9e6c84c7c226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Request method failure captured: Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad HTTP/1.1 Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZaOAL7MvbNmMlJTmhhc7VGRk0MBADDiwy8kC6imnZOcaUe8ctAokL2hWFJNSS55xMf5CW1zXWCW5vLH7KcheFILUFqOHdEf_mC24FyMmscF5Poi9nzdCgQxjRRdkaLsfLErjr9x83SftPYUYpu9j4q5t9G2eEOKGcBWW-eUh6A7O8 Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Openstack-Nova-Api-Version: 2.20 Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: f6606b37accd43619058aef23f10745b Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1618811608 Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:48:16.300823 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:48:16.302725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/f6606b37accd43619058aef23f10745b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/f6606b37accd43619058aef23f10745b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_f6606b37accd43619058aef23f10745b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/f6606b37accd43619058aef23f10745b"}]}] Jul 11 14:48:16.302725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1618811608 Jul 11 14:48:16.302725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: f6606b37accd43619058aef23f10745b Jul 11 14:48:16.302725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1618811608 Jul 11 14:48:16.302725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-AttachVolumeShelveTestJSON-1618811608-project Jul 11 14:48:16.302725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:48:16.302725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:48:16.302725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 99a62e49e6d249c2a169ef7122e762ec Jul 11 14:48:16.302725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1618811608-project Jul 11 14:48:16.302725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:48:16.302725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance 43772fe7-b391-4878-bbea-658602d822ad could not be found. Jul 11 14:48:16.302725 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '43772fe7-b391-4878-bbea-658602d822ad'} Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 43772fe7-b391-4878-bbea-658602d822ad could not be found. Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:48:16.303559 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:48:16.304877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:48:16.304877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:48:16.304877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:48:16.304877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:48:16.304877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:48:16.304877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:48:16.304877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:48:16.304877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 43772fe7-b391-4878-bbea-658602d822ad could not be found. Jul 11 14:48:16.304877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:48:16.304877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:48:16.304877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:48:16.304877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:16.304877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:48:16.304877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:48:16.304877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:48:16.304877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:48:16.304877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 43772fe7-b391-4878-bbea-658602d822ad could not be found. Jul 11 14:48:16.304877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:48:16.307661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-df05b2f4-6579-43f2-a85d-9e6c84c7c226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Returning 404 to user: Instance 43772fe7-b391-4878-bbea-658602d822ad could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:48:16.308873 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-df05b2f4-6579-43f2-a85d-9e6c84c7c226 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 404 len: 111 microversion: 2.20 time: 0.053879 Jul 11 14:48:16.309461 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 514/995] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:48:16 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jul 11 14:48:16.324147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-72f3e961-12ef-4c57-aac7-2b4a98ea0bf1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '.version_select at 0x7f5930218dc0>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:16.329656 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d60a87a5-b9e1-4d99-8a04-fec0d8ed7bc8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:16.347785 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d60a87a5-b9e1-4d99-8a04-fec0d8ed7bc8 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1267 microversion: 2.1 time: 0.259307 Jul 11 14:48:16.349660 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 482/996] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:16 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1267 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:16.355653 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-72f3e961-12ef-4c57-aac7-2b4a98ea0bf1 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-546882754" status: 204 len: 0 microversion: 2.20 time: 0.038711 Jul 11 14:48:16.356636 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 515/997] 10.222.0.72 () {60 vars in 1258 bytes} [Mon Jul 11 14:48:16 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-546882754 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Jul 11 14:48:17.377050 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-fa9245c0-3f69-411e-8611-334eeb3b4b2a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:17.378181 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-fa9245c0-3f69-411e-8611-334eeb3b4b2a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:17.391528 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fa9245c0-3f69-411e-8611-334eeb3b4b2a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:17.392036 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fa9245c0-3f69-411e-8611-334eeb3b4b2a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:17.392912 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fa9245c0-3f69-411e-8611-334eeb3b4b2a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:17.393363 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-fa9245c0-3f69-411e-8611-334eeb3b4b2a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:17.555181 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-fa9245c0-3f69-411e-8611-334eeb3b4b2a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:17.563125 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-fa9245c0-3f69-411e-8611-334eeb3b4b2a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1267 microversion: 2.1 time: 0.194087 Jul 11 14:48:17.563741 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 483/998] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:17 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1267 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:18.581808 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-fdad7c67-07b0-4f16-9dcd-71202d026000 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:18.582364 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-fdad7c67-07b0-4f16-9dcd-71202d026000 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:18.589793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fdad7c67-07b0-4f16-9dcd-71202d026000 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:18.590243 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fdad7c67-07b0-4f16-9dcd-71202d026000 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:18.591053 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fdad7c67-07b0-4f16-9dcd-71202d026000 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:18.591490 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fdad7c67-07b0-4f16-9dcd-71202d026000 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:18.739297 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-fdad7c67-07b0-4f16-9dcd-71202d026000 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:18.747605 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-fdad7c67-07b0-4f16-9dcd-71202d026000 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1267 microversion: 2.1 time: 0.170378 Jul 11 14:48:18.748195 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 516/999] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:18 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1267 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:19.429157 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c420bef8-d9df-4e4c-a0f9-e80fd444e6ce tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:19.429949 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c420bef8-d9df-4e4c-a0f9-e80fd444e6ce tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:19.436585 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c420bef8-d9df-4e4c-a0f9-e80fd444e6ce tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:19.438465 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c420bef8-d9df-4e4c-a0f9-e80fd444e6ce tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:19.439287 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c420bef8-d9df-4e4c-a0f9-e80fd444e6ce tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:19.439960 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c420bef8-d9df-4e4c-a0f9-e80fd444e6ce tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:19.465576 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-c420bef8-d9df-4e4c-a0f9-e80fd444e6ce tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] HTTP exception thrown: Instance 43772fe7-b391-4878-bbea-658602d822ad could not be found. Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c420bef8-d9df-4e4c-a0f9-e80fd444e6ce tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Request method failure captured: Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: DELETE /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad HTTP/1.1 Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDZaOAL7MvbNmMlJTmhhc7VGRk0MBADDiwy8kC6imnZOcaUe8ctAokL2hWFJNSS55xMf5CW1zXWCW5vLH7KcheFILUFqOHdEf_mC24FyMmscF5Poi9nzdCgQxjRRdkaLsfLErjr9x83SftPYUYpu9j4q5t9G2eEOKGcBWW-eUh6A7O8 Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Openstack-Nova-Api-Version: 2.20 Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: f6606b37accd43619058aef23f10745b Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1618811608 Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:48:19.467220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:48:19.470601 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/f6606b37accd43619058aef23f10745b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/f6606b37accd43619058aef23f10745b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_f6606b37accd43619058aef23f10745b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/f6606b37accd43619058aef23f10745b"}]}] Jul 11 14:48:19.470601 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1618811608 Jul 11 14:48:19.470601 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: f6606b37accd43619058aef23f10745b Jul 11 14:48:19.470601 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1618811608 Jul 11 14:48:19.470601 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-AttachVolumeShelveTestJSON-1618811608-project Jul 11 14:48:19.470601 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:48:19.470601 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:48:19.470601 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 99a62e49e6d249c2a169ef7122e762ec Jul 11 14:48:19.470601 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1618811608-project Jul 11 14:48:19.470601 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:48:19.470601 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance 43772fe7-b391-4878-bbea-658602d822ad could not be found. Jul 11 14:48:19.470601 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': '43772fe7-b391-4878-bbea-658602d822ad'} Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 43772fe7-b391-4878-bbea-658602d822ad could not be found. Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:48:19.472013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 43772fe7-b391-4878-bbea-658602d822ad could not be found. Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 43772fe7-b391-4878-bbea-658602d822ad could not be found. Jul 11 14:48:19.474780 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:19.477231 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c420bef8-d9df-4e4c-a0f9-e80fd444e6ce tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Returning 404 to user: Instance 43772fe7-b391-4878-bbea-658602d822ad could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:48:19.477231 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c420bef8-d9df-4e4c-a0f9-e80fd444e6ce tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "DELETE /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 404 len: 111 microversion: 2.20 time: 0.044130 Jul 11 14:48:19.477231 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 484/1000] 10.222.0.72 () {60 vars in 1279 bytes} [Mon Jul 11 14:48:19 2022] DELETE /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jul 11 14:48:19.477707 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-fc12bd8a-6482-4822-97e7-313fbd49f903 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:19.478333 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-fc12bd8a-6482-4822-97e7-313fbd49f903 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 43772fe7-b391-4878-bbea-658602d822ad] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:19.485136 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fc12bd8a-6482-4822-97e7-313fbd49f903 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:19.485588 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fc12bd8a-6482-4822-97e7-313fbd49f903 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:19.486371 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fc12bd8a-6482-4822-97e7-313fbd49f903 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:19.486845 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fc12bd8a-6482-4822-97e7-313fbd49f903 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:19.503055 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-fc12bd8a-6482-4822-97e7-313fbd49f903 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] HTTP exception thrown: Instance 43772fe7-b391-4878-bbea-658602d822ad could not be found. Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-fc12bd8a-6482-4822-97e7-313fbd49f903 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Request method failure captured: Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad HTTP/1.1 Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZaOAL7MvbNmMlJTmhhc7VGRk0MBADDiwy8kC6imnZOcaUe8ctAokL2hWFJNSS55xMf5CW1zXWCW5vLH7KcheFILUFqOHdEf_mC24FyMmscF5Poi9nzdCgQxjRRdkaLsfLErjr9x83SftPYUYpu9j4q5t9G2eEOKGcBWW-eUh6A7O8 Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Openstack-Nova-Api-Version: 2.20 Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: f6606b37accd43619058aef23f10745b Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1618811608 Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:48:19.504761 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:48:19.507588 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/f6606b37accd43619058aef23f10745b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/f6606b37accd43619058aef23f10745b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_f6606b37accd43619058aef23f10745b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/f6606b37accd43619058aef23f10745b"}]}] Jul 11 14:48:19.507588 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1618811608 Jul 11 14:48:19.507588 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: f6606b37accd43619058aef23f10745b Jul 11 14:48:19.507588 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1618811608 Jul 11 14:48:19.507588 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-AttachVolumeShelveTestJSON-1618811608-project Jul 11 14:48:19.507588 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:48:19.507588 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:48:19.507588 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 99a62e49e6d249c2a169ef7122e762ec Jul 11 14:48:19.507588 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1618811608-project Jul 11 14:48:19.507588 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:48:19.507588 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance 43772fe7-b391-4878-bbea-658602d822ad could not be found. Jul 11 14:48:19.507588 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '43772fe7-b391-4878-bbea-658602d822ad'} Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 43772fe7-b391-4878-bbea-658602d822ad could not be found. Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:48:19.509183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:48:19.511727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:48:19.511727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:48:19.511727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:48:19.511727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:48:19.511727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:48:19.511727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:48:19.511727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:48:19.511727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 43772fe7-b391-4878-bbea-658602d822ad could not be found. Jul 11 14:48:19.511727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:48:19.511727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:48:19.511727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:48:19.511727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:19.511727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:48:19.511727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:48:19.511727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:48:19.511727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:48:19.511727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 43772fe7-b391-4878-bbea-658602d822ad could not be found. Jul 11 14:48:19.511727 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:48:19.514213 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-fc12bd8a-6482-4822-97e7-313fbd49f903 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Returning 404 to user: Instance 43772fe7-b391-4878-bbea-658602d822ad could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:48:19.514213 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-fc12bd8a-6482-4822-97e7-313fbd49f903 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad" status: 404 len: 111 microversion: 2.20 time: 0.032422 Jul 11 14:48:19.514213 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 517/1001] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:48:19 2022] GET /compute/v2.1/servers/43772fe7-b391-4878-bbea-658602d822ad => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Jul 11 14:48:19.771896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-b5759cee-5a30-4d8c-b4c5-c78005a9366c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:19.772868 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-b5759cee-5a30-4d8c-b4c5-c78005a9366c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:19.779264 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b5759cee-5a30-4d8c-b4c5-c78005a9366c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:19.780336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b5759cee-5a30-4d8c-b4c5-c78005a9366c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:19.781126 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b5759cee-5a30-4d8c-b4c5-c78005a9366c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:19.782287 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b5759cee-5a30-4d8c-b4c5-c78005a9366c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:20.018573 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-b5759cee-5a30-4d8c-b4c5-c78005a9366c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:20.033887 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-b5759cee-5a30-4d8c-b4c5-c78005a9366c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1267 microversion: 2.1 time: 0.271543 Jul 11 14:48:20.035917 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 485/1002] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:19 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1267 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:20.689812 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-346c431b-19ad-4de7-a387-39484c82f3e6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:20.691051 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-346c431b-19ad-4de7-a387-39484c82f3e6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:20.700971 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-346c431b-19ad-4de7-a387-39484c82f3e6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:20.701424 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-346c431b-19ad-4de7-a387-39484c82f3e6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:20.702247 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-346c431b-19ad-4de7-a387-39484c82f3e6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:20.702735 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-346c431b-19ad-4de7-a387-39484c82f3e6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:20.720450 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-346c431b-19ad-4de7-a387-39484c82f3e6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] HTTP exception thrown: Instance 13da005a-e085-43aa-8a18-1fe85e4e95af could not be found. Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-346c431b-19ad-4de7-a387-39484c82f3e6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Request method failure captured: Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: DELETE /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af HTTP/1.1 Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDZaOAL7MvbNmMlJTmhhc7VGRk0MBADDiwy8kC6imnZOcaUe8ctAokL2hWFJNSS55xMf5CW1zXWCW5vLH7KcheFILUFqOHdEf_mC24FyMmscF5Poi9nzdCgQxjRRdkaLsfLErjr9x83SftPYUYpu9j4q5t9G2eEOKGcBWW-eUh6A7O8 Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Openstack-Nova-Api-Version: 2.20 Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: f6606b37accd43619058aef23f10745b Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1618811608 Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:48:20.722086 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:48:20.723862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/f6606b37accd43619058aef23f10745b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/f6606b37accd43619058aef23f10745b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_f6606b37accd43619058aef23f10745b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/f6606b37accd43619058aef23f10745b"}]}] Jul 11 14:48:20.723862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1618811608 Jul 11 14:48:20.723862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: f6606b37accd43619058aef23f10745b Jul 11 14:48:20.723862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1618811608 Jul 11 14:48:20.723862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-AttachVolumeShelveTestJSON-1618811608-project Jul 11 14:48:20.723862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:48:20.723862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:48:20.723862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 99a62e49e6d249c2a169ef7122e762ec Jul 11 14:48:20.723862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1618811608-project Jul 11 14:48:20.723862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:48:20.723862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance 13da005a-e085-43aa-8a18-1fe85e4e95af could not be found. Jul 11 14:48:20.723862 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '13da005a-e085-43aa-8a18-1fe85e4e95af'} Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 13da005a-e085-43aa-8a18-1fe85e4e95af could not be found. Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:48:20.724849 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 13da005a-e085-43aa-8a18-1fe85e4e95af could not be found. Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 13da005a-e085-43aa-8a18-1fe85e4e95af could not be found. Jul 11 14:48:20.726623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:48:20.728987 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-346c431b-19ad-4de7-a387-39484c82f3e6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Returning 404 to user: Instance 13da005a-e085-43aa-8a18-1fe85e4e95af could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:48:20.730017 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-346c431b-19ad-4de7-a387-39484c82f3e6 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "DELETE /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 404 len: 111 microversion: 2.20 time: 0.046681 Jul 11 14:48:20.730627 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 518/1003] 10.222.0.72 () {60 vars in 1279 bytes} [Mon Jul 11 14:48:20 2022] DELETE /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jul 11 14:48:20.745992 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7bdfb259-c8eb-482b-91b4-04c18e5f4566 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:20.748343 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-7bdfb259-c8eb-482b-91b4-04c18e5f4566 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] [instance: 13da005a-e085-43aa-8a18-1fe85e4e95af] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:20.765483 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7bdfb259-c8eb-482b-91b4-04c18e5f4566 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:20.767296 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7bdfb259-c8eb-482b-91b4-04c18e5f4566 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:20.769044 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7bdfb259-c8eb-482b-91b4-04c18e5f4566 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:20.770173 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-7bdfb259-c8eb-482b-91b4-04c18e5f4566 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:20.791051 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-7bdfb259-c8eb-482b-91b4-04c18e5f4566 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] HTTP exception thrown: Instance 13da005a-e085-43aa-8a18-1fe85e4e95af could not be found. Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7bdfb259-c8eb-482b-91b4-04c18e5f4566 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Request method failure captured: Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af HTTP/1.1 Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDZaOAL7MvbNmMlJTmhhc7VGRk0MBADDiwy8kC6imnZOcaUe8ctAokL2hWFJNSS55xMf5CW1zXWCW5vLH7KcheFILUFqOHdEf_mC24FyMmscF5Poi9nzdCgQxjRRdkaLsfLErjr9x83SftPYUYpu9j4q5t9G2eEOKGcBWW-eUh6A7O8 Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Openstack-Nova-Api-Version: 2.20 Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: f6606b37accd43619058aef23f10745b Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1618811608 Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:48:20.794392 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:48:20.796028 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/f6606b37accd43619058aef23f10745b"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/f6606b37accd43619058aef23f10745b"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_f6606b37accd43619058aef23f10745b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/f6606b37accd43619058aef23f10745b"}]}] Jul 11 14:48:20.796028 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1618811608 Jul 11 14:48:20.796028 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: f6606b37accd43619058aef23f10745b Jul 11 14:48:20.796028 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1618811608 Jul 11 14:48:20.796028 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-AttachVolumeShelveTestJSON-1618811608-project Jul 11 14:48:20.796028 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:48:20.796028 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:48:20.796028 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 99a62e49e6d249c2a169ef7122e762ec Jul 11 14:48:20.796028 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1618811608-project Jul 11 14:48:20.796028 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:48:20.796028 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance 13da005a-e085-43aa-8a18-1fe85e4e95af could not be found. Jul 11 14:48:20.796028 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': '13da005a-e085-43aa-8a18-1fe85e4e95af'} Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 13da005a-e085-43aa-8a18-1fe85e4e95af could not be found. Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:48:20.797374 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:48:20.798900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:48:20.798900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:48:20.798900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:48:20.798900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:48:20.798900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:48:20.798900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:48:20.798900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:48:20.798900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 13da005a-e085-43aa-8a18-1fe85e4e95af could not be found. Jul 11 14:48:20.798900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:20.798900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:48:20.798900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:20.798900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:20.798900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:48:20.798900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:48:20.798900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:48:20.798900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:48:20.798900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 13da005a-e085-43aa-8a18-1fe85e4e95af could not be found. Jul 11 14:48:20.798900 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:20.800268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-7bdfb259-c8eb-482b-91b4-04c18e5f4566 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] Returning 404 to user: Instance 13da005a-e085-43aa-8a18-1fe85e4e95af could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:48:20.800268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-7bdfb259-c8eb-482b-91b4-04c18e5f4566 tempest-AttachVolumeShelveTestJSON-1618811608 tempest-AttachVolumeShelveTestJSON-1618811608-project] 10.222.0.72 "GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af" status: 404 len: 111 microversion: 2.20 time: 0.059421 Jul 11 14:48:20.800789 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 486/1004] 10.222.0.72 () {60 vars in 1276 bytes} [Mon Jul 11 14:48:20 2022] GET /compute/v2.1/servers/13da005a-e085-43aa-8a18-1fe85e4e95af => generated 111 bytes in 64 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Jul 11 14:48:21.063473 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-af8fad49-ec9f-4ce6-ad0f-c4346589aaf7 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:21.066204 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-af8fad49-ec9f-4ce6-ad0f-c4346589aaf7 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:21.078390 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-af8fad49-ec9f-4ce6-ad0f-c4346589aaf7 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:21.079293 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-af8fad49-ec9f-4ce6-ad0f-c4346589aaf7 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:21.080557 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-af8fad49-ec9f-4ce6-ad0f-c4346589aaf7 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:21.081258 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-af8fad49-ec9f-4ce6-ad0f-c4346589aaf7 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:21.499142 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-af8fad49-ec9f-4ce6-ad0f-c4346589aaf7 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:21.520880 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-af8fad49-ec9f-4ce6-ad0f-c4346589aaf7 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1267 microversion: 2.1 time: 0.463591 Jul 11 14:48:21.522192 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 519/1005] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:21 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1267 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:22.545352 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-eefb5729-0046-401e-89ae-0e0a97090c39 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:22.546609 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-eefb5729-0046-401e-89ae-0e0a97090c39 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:22.559634 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eefb5729-0046-401e-89ae-0e0a97090c39 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:22.560463 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eefb5729-0046-401e-89ae-0e0a97090c39 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:22.561949 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eefb5729-0046-401e-89ae-0e0a97090c39 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:22.563353 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eefb5729-0046-401e-89ae-0e0a97090c39 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:22.834034 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-eefb5729-0046-401e-89ae-0e0a97090c39 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:22.854629 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-eefb5729-0046-401e-89ae-0e0a97090c39 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1267 microversion: 2.1 time: 0.316600 Jul 11 14:48:22.870259 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 487/1006] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:22 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1267 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:23.433956 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-35eaffd0-73ff-43ae-9ca9-77e4474fc117 req-568e5219-d53e-4a7b-ac96-5e9760356874 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e2f87e01-f360-4b6f-afde-6a64be9f72f7", "name": "network-vif-plugged", "status": "completed", "tag": "e9f1af2a-fec1-4461-bf84-aca44600c87d"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:48:23.452627 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-35eaffd0-73ff-43ae-9ca9-77e4474fc117 req-568e5219-d53e-4a7b-ac96-5e9760356874 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:23.454419 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-35eaffd0-73ff-43ae-9ca9-77e4474fc117 req-568e5219-d53e-4a7b-ac96-5e9760356874 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:23.472474 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-35eaffd0-73ff-43ae-9ca9-77e4474fc117 req-568e5219-d53e-4a7b-ac96-5e9760356874 service nova] Creating event network-vif-plugged:e9f1af2a-fec1-4461-bf84-aca44600c87d for instance e2f87e01-f360-4b6f-afde-6a64be9f72f7 on hv-cinder-maste Jul 11 14:48:23.478617 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-35eaffd0-73ff-43ae-9ca9-77e4474fc117 req-568e5219-d53e-4a7b-ac96-5e9760356874 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048686 Jul 11 14:48:23.479769 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 520/1007] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:48:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:48:23.878787 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-5b542f72-2d07-4c10-a0ab-36374fc149dd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:23.879547 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-5b542f72-2d07-4c10-a0ab-36374fc149dd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:23.887787 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5b542f72-2d07-4c10-a0ab-36374fc149dd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:23.888243 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5b542f72-2d07-4c10-a0ab-36374fc149dd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:23.889163 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5b542f72-2d07-4c10-a0ab-36374fc149dd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:23.889764 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5b542f72-2d07-4c10-a0ab-36374fc149dd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:24.084699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5b542f72-2d07-4c10-a0ab-36374fc149dd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:24.099626 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-5b542f72-2d07-4c10-a0ab-36374fc149dd tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1267 microversion: 2.1 time: 0.224640 Jul 11 14:48:24.102391 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 488/1008] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:23 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1267 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:25.135623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-6e2945d5-c5f1-4b1a-b399-9e313c2ba088 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:25.138010 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-6e2945d5-c5f1-4b1a-b399-9e313c2ba088 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:25.163011 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6e2945d5-c5f1-4b1a-b399-9e313c2ba088 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:25.164324 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6e2945d5-c5f1-4b1a-b399-9e313c2ba088 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:25.166118 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6e2945d5-c5f1-4b1a-b399-9e313c2ba088 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:25.170124 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6e2945d5-c5f1-4b1a-b399-9e313c2ba088 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:25.521070 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-6e2945d5-c5f1-4b1a-b399-9e313c2ba088 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:25.531000 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-6e2945d5-c5f1-4b1a-b399-9e313c2ba088 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1453 microversion: 2.1 time: 0.406013 Jul 11 14:48:25.531915 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 521/1009] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:25 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1453 bytes in 409 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:25.558776 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c5ccf93b-431a-4602-9013-f1d1b2aac522 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:25.559726 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c5ccf93b-431a-4602-9013-f1d1b2aac522 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:25.567733 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c5ccf93b-431a-4602-9013-f1d1b2aac522 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:25.568382 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c5ccf93b-431a-4602-9013-f1d1b2aac522 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:25.569315 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c5ccf93b-431a-4602-9013-f1d1b2aac522 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:25.569869 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c5ccf93b-431a-4602-9013-f1d1b2aac522 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:25.867634 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-c5ccf93b-431a-4602-9013-f1d1b2aac522 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:25.887876 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c5ccf93b-431a-4602-9013-f1d1b2aac522 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1453 microversion: 2.1 time: 0.334516 Jul 11 14:48:25.889484 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 489/1010] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:25 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1453 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:29.066705 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-6cfbdbf7-0e44-4d5b-b04e-a9e64ee2314b tempest-VolumesGetTestJSON-1768733234 tempest-VolumesGetTestJSON-1768733234-project] Action: 'create', calling method: .version_select at 0x7f59300cc8b0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1929305908", "metadata": {"Type": "work"}}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:48:29.112942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-6660b3ba-2729-486c-b590-fbe8bc220d60 req-9c57f501-6ad9-4489-831f-89d64e15d46c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e2f87e01-f360-4b6f-afde-6a64be9f72f7", "tag": "e9f1af2a-fec1-4461-bf84-aca44600c87d"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:48:29.130249 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-6660b3ba-2729-486c-b590-fbe8bc220d60 req-9c57f501-6ad9-4489-831f-89d64e15d46c service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:29.131845 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-6660b3ba-2729-486c-b590-fbe8bc220d60 req-9c57f501-6ad9-4489-831f-89d64e15d46c service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:29.166662 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-6660b3ba-2729-486c-b590-fbe8bc220d60 req-9c57f501-6ad9-4489-831f-89d64e15d46c service nova] Creating event network-changed:e9f1af2a-fec1-4461-bf84-aca44600c87d for instance e2f87e01-f360-4b6f-afde-6a64be9f72f7 on hv-cinder-maste Jul 11 14:48:29.177648 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-6660b3ba-2729-486c-b590-fbe8bc220d60 req-9c57f501-6ad9-4489-831f-89d64e15d46c service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072088 Jul 11 14:48:29.183852 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 490/1011] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:48:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:48:29.684882 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-6cfbdbf7-0e44-4d5b-b04e-a9e64ee2314b tempest-VolumesGetTestJSON-1768733234 tempest-VolumesGetTestJSON-1768733234-project] 10.222.0.72 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 0.875010 Jul 11 14:48:29.686221 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 522/1012] 10.222.0.72 () {60 vars in 1155 bytes} [Mon Jul 11 14:48:28 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 877 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Jul 11 14:48:29.703349 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-92f7ee6f-3802-4cad-bba3-48dfe4d6e506 tempest-VolumesGetTestJSON-1768733234 tempest-VolumesGetTestJSON-1768733234-project] Calling method '.version_select at 0x7f592b7e1550>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:29.912476 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-92f7ee6f-3802-4cad-bba3-48dfe4d6e506 tempest-VolumesGetTestJSON-1768733234 tempest-VolumesGetTestJSON-1768733234-project] 10.222.0.72 "GET /compute/v2.1/os-volumes/c5dd3a53-ffed-4d73-8f55-e990fef89966" status: 200 len: 353 microversion: 2.1 time: 0.217896 Jul 11 14:48:29.915259 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 491/1013] 10.222.0.72 () {58 vars in 1244 bytes} [Mon Jul 11 14:48:29 2022] GET /compute/v2.1/os-volumes/c5dd3a53-ffed-4d73-8f55-e990fef89966 => generated 353 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:48:30.931733 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f2b68e87-e8e8-4000-bea6-e84e3b0c812e tempest-VolumesGetTestJSON-1768733234 tempest-VolumesGetTestJSON-1768733234-project] Calling method '.version_select at 0x7f59303bfdc0>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:30.999281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f2b68e87-e8e8-4000-bea6-e84e3b0c812e tempest-VolumesGetTestJSON-1768733234 tempest-VolumesGetTestJSON-1768733234-project] 10.222.0.72 "GET /compute/v2.1/os-volumes/c5dd3a53-ffed-4d73-8f55-e990fef89966" status: 200 len: 354 microversion: 2.1 time: 0.072208 Jul 11 14:48:31.000529 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 523/1014] 10.222.0.72 () {58 vars in 1244 bytes} [Mon Jul 11 14:48:30 2022] GET /compute/v2.1/os-volumes/c5dd3a53-ffed-4d73-8f55-e990fef89966 => generated 354 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:48:31.012988 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-b73fd7aa-bcfb-4d5b-bd9d-299ae1d392fe tempest-VolumesGetTestJSON-1768733234 tempest-VolumesGetTestJSON-1768733234-project] Calling method '.version_select at 0x7f592b748c10>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:31.065517 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-b73fd7aa-bcfb-4d5b-bd9d-299ae1d392fe tempest-VolumesGetTestJSON-1768733234 tempest-VolumesGetTestJSON-1768733234-project] 10.222.0.72 "GET /compute/v2.1/os-volumes/c5dd3a53-ffed-4d73-8f55-e990fef89966" status: 200 len: 354 microversion: 2.1 time: 0.057142 Jul 11 14:48:31.066088 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 492/1015] 10.222.0.72 () {58 vars in 1244 bytes} [Mon Jul 11 14:48:31 2022] GET /compute/v2.1/os-volumes/c5dd3a53-ffed-4d73-8f55-e990fef89966 => generated 354 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:48:31.083642 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-be318c46-3d13-4efa-8574-433e516a64dd tempest-VolumesGetTestJSON-1768733234 tempest-VolumesGetTestJSON-1768733234-project] Calling method '.version_select at 0x7f59303bfb80>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:31.156073 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-be318c46-3d13-4efa-8574-433e516a64dd tempest-VolumesGetTestJSON-1768733234 tempest-VolumesGetTestJSON-1768733234-project] 10.222.0.72 "DELETE /compute/v2.1/os-volumes/c5dd3a53-ffed-4d73-8f55-e990fef89966" status: 202 len: 0 microversion: 2.1 time: 0.081279 Jul 11 14:48:31.157160 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 524/1016] 10.222.0.72 () {58 vars in 1247 bytes} [Mon Jul 11 14:48:31 2022] DELETE /compute/v2.1/os-volumes/c5dd3a53-ffed-4d73-8f55-e990fef89966 => generated 0 bytes in 84 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:48:31.166138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-83c3be5a-e0a6-4e3d-a184-dec77388956c tempest-VolumesGetTestJSON-1768733234 tempest-VolumesGetTestJSON-1768733234-project] Calling method '.version_select at 0x7f593019e1f0>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:31.228935 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-83c3be5a-e0a6-4e3d-a184-dec77388956c tempest-VolumesGetTestJSON-1768733234 tempest-VolumesGetTestJSON-1768733234-project] 10.222.0.72 "GET /compute/v2.1/os-volumes/c5dd3a53-ffed-4d73-8f55-e990fef89966" status: 200 len: 353 microversion: 2.1 time: 0.066977 Jul 11 14:48:31.229786 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 493/1017] 10.222.0.72 () {58 vars in 1244 bytes} [Mon Jul 11 14:48:31 2022] GET /compute/v2.1/os-volumes/c5dd3a53-ffed-4d73-8f55-e990fef89966 => generated 353 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:48:31.315141 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-8bc5a06c-f1a4-43a6-b0bc-a31cb45968f2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1506162340", "imageRef": "a588fa24-23b5-4e53-87c6-7e600ee3a5ef", "flavorRef": "452", "networks": [{"uuid": "558e7e14-28fa-468b-8d85-62c3486f900d"}]}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:48:31.392894 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.network.neutron [None req-8bc5a06c-f1a4-43a6-b0bc-a31cb45968f2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] validate_networks() for [('558e7e14-28fa-468b-8d85-62c3486f900d', None, None, None, None, None)] {{(pid=130159) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:48:32.178129 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-8bc5a06c-f1a4-43a6-b0bc-a31cb45968f2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Flavor limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:48:32.178919 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-8bc5a06c-f1a4-43a6-b0bc-a31cb45968f2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Image limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:48:32.179202 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-8bc5a06c-f1a4-43a6-b0bc-a31cb45968f2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Flavor pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:48:32.179670 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-8bc5a06c-f1a4-43a6-b0bc-a31cb45968f2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Image pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:48:32.180143 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-8bc5a06c-f1a4-43a6-b0bc-a31cb45968f2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:48:32.191448 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8bc5a06c-f1a4-43a6-b0bc-a31cb45968f2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:32.192223 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8bc5a06c-f1a4-43a6-b0bc-a31cb45968f2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:32.193243 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8bc5a06c-f1a4-43a6-b0bc-a31cb45968f2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:32.193930 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-8bc5a06c-f1a4-43a6-b0bc-a31cb45968f2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:32.217158 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-8bc5a06c-f1a4-43a6-b0bc-a31cb45968f2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Getting quotas for project 23a54d0a3442474db1829cb42e2b3c95. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:48:32.221918 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-8bc5a06c-f1a4-43a6-b0bc-a31cb45968f2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Getting quotas for user 6e9ce7b0fdbd43f28d38a4b5657a3e2e and project 23a54d0a3442474db1829cb42e2b3c95. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:48:32.235920 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8bc5a06c-f1a4-43a6-b0bc-a31cb45968f2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Going to run 1 instances... {{(pid=130159) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:48:32.238469 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-8bc5a06c-f1a4-43a6-b0bc-a31cb45968f2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: bddb57e6-3dc8-4a9a-a551-4727ff237620] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a588fa24-23b5-4e53-87c6-7e600ee3a5ef',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130159) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:48:32.244406 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9fda1614-3dac-4786-a43f-85af50db2da4 tempest-VolumesGetTestJSON-1768733234 tempest-VolumesGetTestJSON-1768733234-project] Calling method '.version_select at 0x7f592b7e1670>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:32.281902 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-9fda1614-3dac-4786-a43f-85af50db2da4 tempest-VolumesGetTestJSON-1768733234 tempest-VolumesGetTestJSON-1768733234-project] HTTP exception thrown: Volume c5dd3a53-ffed-4d73-8f55-e990fef89966 could not be found. Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9fda1614-3dac-4786-a43f-85af50db2da4 tempest-VolumesGetTestJSON-1768733234 tempest-VolumesGetTestJSON-1768733234-project] Request method failure captured: Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/os-volumes/c5dd3a53-ffed-4d73-8f55-e990fef89966 HTTP/1.1 Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDg8d2ffJOAbYwEWJHzq2jrC18wtktktkFKxltWdvl2m9iNIm3SbZYkcz7IqeMKZ_1xChgGXvgurChAt3RIr6HLKq3DeNVRnz9Ba8wZuj4dYBtZQd6fs6jbB9To0Y-VjRNFLD_qvynX1wso2jjc9GUmSu66tkv_C5-Ig1-Z-rV2RWIs Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: 8bb9c3c7a1e54ced9dd9a1d40e736cb4 Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-VolumesGetTestJSON-1768733234 Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:48:32.284325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:48:32.287211 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/8bb9c3c7a1e54ced9dd9a1d40e736cb4"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/8bb9c3c7a1e54ced9dd9a1d40e736cb4"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_8bb9c3c7a1e54ced9dd9a1d40e736cb4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/8bb9c3c7a1e54ced9dd9a1d40e736cb4"}]}] Jul 11 14:48:32.287211 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-VolumesGetTestJSON-1768733234 Jul 11 14:48:32.287211 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: 8bb9c3c7a1e54ced9dd9a1d40e736cb4 Jul 11 14:48:32.287211 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-VolumesGetTestJSON-1768733234 Jul 11 14:48:32.287211 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-VolumesGetTestJSON-1768733234-project Jul 11 14:48:32.287211 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:48:32.287211 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:48:32.287211 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: e4c1e471f9604b269ea04ac173504e7f Jul 11 14:48:32.287211 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-VolumesGetTestJSON-1768733234-project Jul 11 14:48:32.287211 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: Jul 11 14:48:32.287211 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Volume c5dd3a53-ffed-4d73-8f55-e990fef89966 could not be found. Jul 11 14:48:32.287211 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': 'c5dd3a53-ffed-4d73-8f55-e990fef89966'} Jul 11 14:48:32.287211 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume c5dd3a53-ffed-4d73-8f55-e990fef89966 could not be found. (HTTP 404) (Request-ID: req-81d4b967-2e94-4a88-b37f-a92c7bbee2d8) Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:48:32.291553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi item = cinderclient( Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Jul 11 14:48:32.295622 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume c5dd3a53-ffed-4d73-8f55-e990fef89966 could not be found. Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Jul 11 14:48:32.299805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:48:32.302597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume c5dd3a53-ffed-4d73-8f55-e990fef89966 could not be found. Jul 11 14:48:32.302597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:32.302597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:48:32.302597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:32.302597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:32.302597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:48:32.302597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:48:32.302597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:48:32.302597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:48:32.302597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume c5dd3a53-ffed-4d73-8f55-e990fef89966 could not be found. Jul 11 14:48:32.302597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:32.302597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9fda1614-3dac-4786-a43f-85af50db2da4 tempest-VolumesGetTestJSON-1768733234 tempest-VolumesGetTestJSON-1768733234-project] Returning 404 to user: Volume c5dd3a53-ffed-4d73-8f55-e990fef89966 could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:48:32.302597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9fda1614-3dac-4786-a43f-85af50db2da4 tempest-VolumesGetTestJSON-1768733234 tempest-VolumesGetTestJSON-1768733234-project] 10.222.0.72 "GET /compute/v2.1/os-volumes/c5dd3a53-ffed-4d73-8f55-e990fef89966" status: 404 len: 109 microversion: 2.1 time: 0.046600 Jul 11 14:48:32.302597 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 494/1018] 10.222.0.72 () {58 vars in 1244 bytes} [Mon Jul 11 14:48:32 2022] GET /compute/v2.1/os-volumes/c5dd3a53-ffed-4d73-8f55-e990fef89966 => generated 109 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 11 14:48:32.305332 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-8bc5a06c-f1a4-43a6-b0bc-a31cb45968f2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.981046 Jul 11 14:48:32.305332 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 525/1019] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:48:31 2022] POST /compute/v2.1/servers => generated 380 bytes in 983 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 11 14:48:32.314100 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-bd1965b1-0105-46d6-81d4-a20d8f85e1f6 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:32.315506 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-bd1965b1-0105-46d6-81d4-a20d8f85e1f6 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: bddb57e6-3dc8-4a9a-a551-4727ff237620] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:32.406706 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-bd1965b1-0105-46d6-81d4-a20d8f85e1f6 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:32.418130 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-bd1965b1-0105-46d6-81d4-a20d8f85e1f6 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620" status: 200 len: 1214 microversion: 2.1 time: 0.111319 Jul 11 14:48:32.419107 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 495/1020] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:32 2022] GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620 => generated 1214 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:33.437354 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-86248001-6d2f-4f69-8ce2-5cd35cf974ba tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:33.438229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-86248001-6d2f-4f69-8ce2-5cd35cf974ba tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: bddb57e6-3dc8-4a9a-a551-4727ff237620] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:33.459091 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-86248001-6d2f-4f69-8ce2-5cd35cf974ba tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:33.460872 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-86248001-6d2f-4f69-8ce2-5cd35cf974ba tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:33.466019 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-86248001-6d2f-4f69-8ce2-5cd35cf974ba tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:33.468917 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-86248001-6d2f-4f69-8ce2-5cd35cf974ba tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:33.597660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-86248001-6d2f-4f69-8ce2-5cd35cf974ba tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:33.608122 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-86248001-6d2f-4f69-8ce2-5cd35cf974ba tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620" status: 200 len: 1274 microversion: 2.1 time: 0.175073 Jul 11 14:48:33.609500 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 526/1021] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:33 2022] GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620 => generated 1274 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:48:34.641694 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-6cede79d-95d1-4413-b8d4-8e69a41fbde4 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:34.642818 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-6cede79d-95d1-4413-b8d4-8e69a41fbde4 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: bddb57e6-3dc8-4a9a-a551-4727ff237620] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:34.666024 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6cede79d-95d1-4413-b8d4-8e69a41fbde4 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:34.670870 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6cede79d-95d1-4413-b8d4-8e69a41fbde4 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:34.672546 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6cede79d-95d1-4413-b8d4-8e69a41fbde4 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:34.674226 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-6cede79d-95d1-4413-b8d4-8e69a41fbde4 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:35.168818 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-6cede79d-95d1-4413-b8d4-8e69a41fbde4 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:35.195882 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-6cede79d-95d1-4413-b8d4-8e69a41fbde4 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620" status: 200 len: 1314 microversion: 2.1 time: 0.561177 Jul 11 14:48:35.204100 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 496/1022] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:34 2022] GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620 => generated 1314 bytes in 571 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:35.762472 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-9b49d7f4-8cfc-44d3-9f76-df44c733d57f req-14b48080-204a-48cf-9f48-a7585ba287a6 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bddb57e6-3dc8-4a9a-a551-4727ff237620", "tag": "0ae0efa3-a3a2-4fa2-957d-d2b7715eef7a"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:48:35.784283 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-9b49d7f4-8cfc-44d3-9f76-df44c733d57f req-14b48080-204a-48cf-9f48-a7585ba287a6 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:35.785281 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-9b49d7f4-8cfc-44d3-9f76-df44c733d57f req-14b48080-204a-48cf-9f48-a7585ba287a6 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:35.817815 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-9b49d7f4-8cfc-44d3-9f76-df44c733d57f req-14b48080-204a-48cf-9f48-a7585ba287a6 service nova] Creating event network-changed:0ae0efa3-a3a2-4fa2-957d-d2b7715eef7a for instance bddb57e6-3dc8-4a9a-a551-4727ff237620 on hv-cinder-maste Jul 11 14:48:35.831475 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-9b49d7f4-8cfc-44d3-9f76-df44c733d57f req-14b48080-204a-48cf-9f48-a7585ba287a6 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.075208 Jul 11 14:48:35.833007 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 527/1023] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:48:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:48:36.226771 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-d6bf051d-11a5-4f36-ae97-28ee997a7660 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:36.227907 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-d6bf051d-11a5-4f36-ae97-28ee997a7660 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: bddb57e6-3dc8-4a9a-a551-4727ff237620] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:36.241642 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d6bf051d-11a5-4f36-ae97-28ee997a7660 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:36.242415 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d6bf051d-11a5-4f36-ae97-28ee997a7660 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:36.245004 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d6bf051d-11a5-4f36-ae97-28ee997a7660 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:36.246566 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-d6bf051d-11a5-4f36-ae97-28ee997a7660 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:36.301928 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-971b84eb-ab40-4d15-a9ce-4e2e8d88775a tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] Calling method '.version_select at 0x7f59300cc940>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:36.391531 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-971b84eb-ab40-4d15-a9ce-4e2e8d88775a tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] 10.222.0.72 "GET /compute/v2.1/os-volumes/a6ab2a3b-5db0-4ed5-b7ad-eaf0a30ec173" status: 200 len: 351 microversion: 2.1 time: 0.094341 Jul 11 14:48:36.392570 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 528/1024] 10.222.0.72 () {58 vars in 1244 bytes} [Mon Jul 11 14:48:36 2022] GET /compute/v2.1/os-volumes/a6ab2a3b-5db0-4ed5-b7ad-eaf0a30ec173 => generated 351 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:48:36.496355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-d6bf051d-11a5-4f36-ae97-28ee997a7660 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:36.510738 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-d6bf051d-11a5-4f36-ae97-28ee997a7660 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620" status: 200 len: 1314 microversion: 2.1 time: 0.291917 Jul 11 14:48:36.511936 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 497/1025] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:36 2022] GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620 => generated 1314 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:37.531532 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-bcba811c-ea7b-4a6b-a6f3-3207a5e9027e tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:37.532431 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-bcba811c-ea7b-4a6b-a6f3-3207a5e9027e tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: bddb57e6-3dc8-4a9a-a551-4727ff237620] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:37.539160 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bcba811c-ea7b-4a6b-a6f3-3207a5e9027e tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:37.539746 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bcba811c-ea7b-4a6b-a6f3-3207a5e9027e tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:37.540565 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bcba811c-ea7b-4a6b-a6f3-3207a5e9027e tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:37.541013 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-bcba811c-ea7b-4a6b-a6f3-3207a5e9027e tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:37.755416 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-bcba811c-ea7b-4a6b-a6f3-3207a5e9027e tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:37.765935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-bcba811c-ea7b-4a6b-a6f3-3207a5e9027e tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620" status: 200 len: 1314 microversion: 2.1 time: 0.238681 Jul 11 14:48:37.766695 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 529/1026] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:37 2022] GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620 => generated 1314 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:37.883493 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-780023da-bf4a-48bc-90fb-68cdcc4594de tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] Calling method '.version_select at 0x7f593019e940>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:37.936625 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-780023da-bf4a-48bc-90fb-68cdcc4594de tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] 10.222.0.72 "GET /compute/v2.1/os-volumes/24f49baa-72d6-4535-8c06-d0648d82fe47" status: 200 len: 351 microversion: 2.1 time: 0.057264 Jul 11 14:48:37.937696 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 498/1027] 10.222.0.72 () {58 vars in 1244 bytes} [Mon Jul 11 14:48:37 2022] GET /compute/v2.1/os-volumes/24f49baa-72d6-4535-8c06-d0648d82fe47 => generated 351 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:48:38.788932 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3aa24645-5477-4202-b0f1-4381ff1a5045 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:38.789900 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-3aa24645-5477-4202-b0f1-4381ff1a5045 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: bddb57e6-3dc8-4a9a-a551-4727ff237620] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:38.797571 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3aa24645-5477-4202-b0f1-4381ff1a5045 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:38.798020 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3aa24645-5477-4202-b0f1-4381ff1a5045 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:38.798865 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3aa24645-5477-4202-b0f1-4381ff1a5045 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:38.799312 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-3aa24645-5477-4202-b0f1-4381ff1a5045 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:39.023231 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-3aa24645-5477-4202-b0f1-4381ff1a5045 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:39.045212 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3aa24645-5477-4202-b0f1-4381ff1a5045 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620" status: 200 len: 1314 microversion: 2.1 time: 0.260282 Jul 11 14:48:39.046388 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 530/1028] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:38 2022] GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620 => generated 1314 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:48:39.344887 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-316370eb-0ad4-4f9c-94ab-4f74cb9b8e25 tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] Calling method '.version_select at 0x7f592b7c8ee0>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:39.438785 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-316370eb-0ad4-4f9c-94ab-4f74cb9b8e25 tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] 10.222.0.72 "GET /compute/v2.1/os-volumes/f0fbb64c-d1d4-4d18-b966-674ed603c48a" status: 200 len: 350 microversion: 2.1 time: 0.114218 Jul 11 14:48:39.439996 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 499/1029] 10.222.0.72 () {58 vars in 1244 bytes} [Mon Jul 11 14:48:39 2022] GET /compute/v2.1/os-volumes/f0fbb64c-d1d4-4d18-b966-674ed603c48a => generated 350 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:48:39.458606 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3d038c30-01d2-4dae-a5f5-2ef91427b255 tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] Calling method '.version_select at 0x7f5930317e50>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:39.618379 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3d038c30-01d2-4dae-a5f5-2ef91427b255 tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] 10.222.0.72 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.166069 Jul 11 14:48:39.619255 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 531/1030] 10.222.0.72 () {58 vars in 1133 bytes} [Mon Jul 11 14:48:39 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:39.634573 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-ea3e0774-1f62-48bb-a653-2aa91313bdd8 tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] Calling method '.version_select at 0x7f593017eb80>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:39.758892 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-ea3e0774-1f62-48bb-a653-2aa91313bdd8 tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] 10.222.0.72 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.127759 Jul 11 14:48:39.759980 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 500/1031] 10.222.0.72 () {58 vars in 1148 bytes} [Mon Jul 11 14:48:39 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:48:39.777563 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a0df66ec-a9be-4157-abce-5c5c30451054 tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] Calling method '.version_select at 0x7f59303093a0>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:39.917110 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a0df66ec-a9be-4157-abce-5c5c30451054 tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] 10.222.0.72 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.145608 Jul 11 14:48:39.920552 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 532/1032] 10.222.0.72 () {58 vars in 1133 bytes} [Mon Jul 11 14:48:39 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:39.933651 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8723d30e-e88f-43f2-9f41-0f5350724943 tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] Calling method '.version_select at 0x7f5930158280>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:40.034525 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8723d30e-e88f-43f2-9f41-0f5350724943 tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] 10.222.0.72 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.107308 Jul 11 14:48:40.039125 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 501/1033] 10.222.0.72 () {58 vars in 1166 bytes} [Mon Jul 11 14:48:39 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:48:40.050853 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-3fd31ef3-a0be-4231-baf0-0bccd1f87958 tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] Calling method '.version_select at 0x7f593010af70>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:40.085581 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-719bc08e-15a2-4655-8027-077629726624 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:40.089535 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-719bc08e-15a2-4655-8027-077629726624 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: bddb57e6-3dc8-4a9a-a551-4727ff237620] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:40.105922 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-719bc08e-15a2-4655-8027-077629726624 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:40.106914 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-719bc08e-15a2-4655-8027-077629726624 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:40.107998 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-719bc08e-15a2-4655-8027-077629726624 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:40.109001 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-719bc08e-15a2-4655-8027-077629726624 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:40.168847 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-3fd31ef3-a0be-4231-baf0-0bccd1f87958 tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] 10.222.0.72 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.122783 Jul 11 14:48:40.172431 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 533/1034] 10.222.0.72 () {58 vars in 1169 bytes} [Mon Jul 11 14:48:40 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:48:40.191299 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-62120dff-252e-4969-8ab3-954b0c5fa789 tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] Calling method '.version_select at 0x7f5930269430>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:40.305914 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-62120dff-252e-4969-8ab3-954b0c5fa789 tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] 10.222.0.72 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.121071 Jul 11 14:48:40.308291 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 534/1035] 10.222.0.72 () {58 vars in 1154 bytes} [Mon Jul 11 14:48:40 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:48:40.319966 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9b811cf4-9d93-4e21-ba17-c73d008e69dc tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] Calling method '.version_select at 0x7f59304c5af0>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:40.339963 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-719bc08e-15a2-4655-8027-077629726624 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:40.350952 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-719bc08e-15a2-4655-8027-077629726624 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620" status: 200 len: 1314 microversion: 2.1 time: 0.279647 Jul 11 14:48:40.352021 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 502/1036] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:40 2022] GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620 => generated 1314 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:40.414078 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9b811cf4-9d93-4e21-ba17-c73d008e69dc tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] 10.222.0.72 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.098424 Jul 11 14:48:40.414837 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 535/1037] 10.222.0.72 () {58 vars in 1187 bytes} [Mon Jul 11 14:48:40 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:48:40.434505 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-95957c15-d649-4e3c-bb2c-c397580d438a tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] Calling method '.version_select at 0x7f592b7c8940>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:40.535044 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-95957c15-d649-4e3c-bb2c-c397580d438a tempest-VolumesTestJSON-1575798818 tempest-VolumesTestJSON-1575798818-project] 10.222.0.72 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.106435 Jul 11 14:48:40.535632 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 503/1038] 10.222.0.72 () {58 vars in 1154 bytes} [Mon Jul 11 14:48:40 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:48:41.377971 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-0372201f-6627-4c71-b249-1eece821fa6c tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:41.379479 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-0372201f-6627-4c71-b249-1eece821fa6c tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: bddb57e6-3dc8-4a9a-a551-4727ff237620] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:41.391585 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0372201f-6627-4c71-b249-1eece821fa6c tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:41.392503 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0372201f-6627-4c71-b249-1eece821fa6c tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:41.393768 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0372201f-6627-4c71-b249-1eece821fa6c tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:41.394671 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0372201f-6627-4c71-b249-1eece821fa6c tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:41.499553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-0e67a852-dc41-4b84-8c52-f8c51f17a95d req-fb6e1a55-c60f-4665-b6b1-b3fea4c58730 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bddb57e6-3dc8-4a9a-a551-4727ff237620", "name": "network-vif-plugged", "status": "completed", "tag": "0ae0efa3-a3a2-4fa2-957d-d2b7715eef7a"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:48:41.526101 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-0e67a852-dc41-4b84-8c52-f8c51f17a95d req-fb6e1a55-c60f-4665-b6b1-b3fea4c58730 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:41.527321 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-0e67a852-dc41-4b84-8c52-f8c51f17a95d req-fb6e1a55-c60f-4665-b6b1-b3fea4c58730 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:41.543721 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-0e67a852-dc41-4b84-8c52-f8c51f17a95d req-fb6e1a55-c60f-4665-b6b1-b3fea4c58730 service nova] Creating event network-vif-plugged:0ae0efa3-a3a2-4fa2-957d-d2b7715eef7a for instance bddb57e6-3dc8-4a9a-a551-4727ff237620 on hv-cinder-maste Jul 11 14:48:41.553464 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-0e67a852-dc41-4b84-8c52-f8c51f17a95d req-fb6e1a55-c60f-4665-b6b1-b3fea4c58730 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062182 Jul 11 14:48:41.554724 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 504/1039] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:48:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:48:41.613496 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0372201f-6627-4c71-b249-1eece821fa6c tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:41.628296 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-0372201f-6627-4c71-b249-1eece821fa6c tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620" status: 200 len: 1314 microversion: 2.1 time: 0.258261 Jul 11 14:48:41.629389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 536/1040] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:41 2022] GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620 => generated 1314 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:42.650614 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=2 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:48:42.656242 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-81b7c9a1-484b-4fb6-9b60-505fc2b5c948 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:42.657043 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-81b7c9a1-484b-4fb6-9b60-505fc2b5c948 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: bddb57e6-3dc8-4a9a-a551-4727ff237620] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:42.664271 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-81b7c9a1-484b-4fb6-9b60-505fc2b5c948 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:42.664718 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-81b7c9a1-484b-4fb6-9b60-505fc2b5c948 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:42.665497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-81b7c9a1-484b-4fb6-9b60-505fc2b5c948 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:42.665930 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-81b7c9a1-484b-4fb6-9b60-505fc2b5c948 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:42.847541 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-81b7c9a1-484b-4fb6-9b60-505fc2b5c948 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:42.855703 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-81b7c9a1-484b-4fb6-9b60-505fc2b5c948 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620" status: 200 len: 1314 microversion: 2.1 time: 0.206009 Jul 11 14:48:42.856462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 505/1041] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:42 2022] GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620 => generated 1314 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:43.873734 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-0964ad91-c8ac-472a-80ae-9b529ac5e5c9 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:43.874473 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-0964ad91-c8ac-472a-80ae-9b529ac5e5c9 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: bddb57e6-3dc8-4a9a-a551-4727ff237620] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:43.876846 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=2 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:48:43.877515 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=6 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:48:43.884052 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0964ad91-c8ac-472a-80ae-9b529ac5e5c9 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:43.884502 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0964ad91-c8ac-472a-80ae-9b529ac5e5c9 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:43.885275 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0964ad91-c8ac-472a-80ae-9b529ac5e5c9 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:43.885708 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0964ad91-c8ac-472a-80ae-9b529ac5e5c9 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:44.066611 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0964ad91-c8ac-472a-80ae-9b529ac5e5c9 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:44.077029 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-0964ad91-c8ac-472a-80ae-9b529ac5e5c9 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620" status: 200 len: 1499 microversion: 2.1 time: 0.207435 Jul 11 14:48:44.078246 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 537/1042] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:43 2022] GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620 => generated 1499 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:44.630227 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-ec55a42a-1e0c-4b41-a1ec-18694d3226be tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:44.631114 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-ec55a42a-1e0c-4b41-a1ec-18694d3226be tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: bddb57e6-3dc8-4a9a-a551-4727ff237620] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:44.639226 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ec55a42a-1e0c-4b41-a1ec-18694d3226be tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:44.640485 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ec55a42a-1e0c-4b41-a1ec-18694d3226be tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:44.641497 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ec55a42a-1e0c-4b41-a1ec-18694d3226be tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:44.642123 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ec55a42a-1e0c-4b41-a1ec-18694d3226be tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:44.690275 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-ec55a42a-1e0c-4b41-a1ec-18694d3226be tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: bddb57e6-3dc8-4a9a-a551-4727ff237620] Going to try to terminate instance {{(pid=130160) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:48:44.838636 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-ec55a42a-1e0c-4b41-a1ec-18694d3226be tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "DELETE /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620" status: 204 len: 0 microversion: 2.1 time: 0.212521 Jul 11 14:48:44.839354 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 506/1043] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:48:44 2022] DELETE /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 11 14:48:44.852402 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a86b09d5-8abd-49d9-bfdd-64d820f5fce0 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:44.853589 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a86b09d5-8abd-49d9-bfdd-64d820f5fce0 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: bddb57e6-3dc8-4a9a-a551-4727ff237620] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:44.865817 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a86b09d5-8abd-49d9-bfdd-64d820f5fce0 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:44.866691 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a86b09d5-8abd-49d9-bfdd-64d820f5fce0 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:44.868096 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a86b09d5-8abd-49d9-bfdd-64d820f5fce0 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:44.868899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a86b09d5-8abd-49d9-bfdd-64d820f5fce0 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:44.912624 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=68 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:48:45.114353 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a86b09d5-8abd-49d9-bfdd-64d820f5fce0 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:45.130021 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a86b09d5-8abd-49d9-bfdd-64d820f5fce0 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620" status: 200 len: 1505 microversion: 2.1 time: 0.285230 Jul 11 14:48:45.131147 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 538/1044] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:44 2022] GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620 => generated 1505 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:48:45.588607 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 11 14:48:45.588951 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Mon Jul 11 14:48:45 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 11 14:48:46.152003 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-4438923a-4480-40df-b0a7-e8f36cdc82c2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:46.152728 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-4438923a-4480-40df-b0a7-e8f36cdc82c2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: bddb57e6-3dc8-4a9a-a551-4727ff237620] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:46.156298 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=39,nova_api:UPDATE=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:48:46.163336 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4438923a-4480-40df-b0a7-e8f36cdc82c2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:46.163879 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4438923a-4480-40df-b0a7-e8f36cdc82c2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:46.164811 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4438923a-4480-40df-b0a7-e8f36cdc82c2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:46.165325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-4438923a-4480-40df-b0a7-e8f36cdc82c2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:46.172420 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=82,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:48:46.350911 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-4438923a-4480-40df-b0a7-e8f36cdc82c2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:46.364082 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-4438923a-4480-40df-b0a7-e8f36cdc82c2 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620" status: 200 len: 1505 microversion: 2.1 time: 0.217315 Jul 11 14:48:46.365160 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 507/1045] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:46 2022] GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620 => generated 1505 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:48:47.384786 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-d24ccdae-bbb5-49ba-b7b9-0cfe3285286b tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:47.385827 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-d24ccdae-bbb5-49ba-b7b9-0cfe3285286b tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: bddb57e6-3dc8-4a9a-a551-4727ff237620] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:47.392057 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=48,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:48:47.400095 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d24ccdae-bbb5-49ba-b7b9-0cfe3285286b tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:47.400863 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d24ccdae-bbb5-49ba-b7b9-0cfe3285286b tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:47.402508 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d24ccdae-bbb5-49ba-b7b9-0cfe3285286b tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:47.403681 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-d24ccdae-bbb5-49ba-b7b9-0cfe3285286b tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:47.745332 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-882754b5-d0a8-41e6-b1f3-b9cf1e33998e req-18f3e563-f552-42e1-ba29-e51313bd334f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bddb57e6-3dc8-4a9a-a551-4727ff237620", "name": "network-vif-deleted", "tag": "0ae0efa3-a3a2-4fa2-957d-d2b7715eef7a"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:48:47.755820 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-882754b5-d0a8-41e6-b1f3-b9cf1e33998e req-18f3e563-f552-42e1-ba29-e51313bd334f service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:47.756550 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-882754b5-d0a8-41e6-b1f3-b9cf1e33998e req-18f3e563-f552-42e1-ba29-e51313bd334f service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:47.777818 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-882754b5-d0a8-41e6-b1f3-b9cf1e33998e req-18f3e563-f552-42e1-ba29-e51313bd334f service nova] Creating event network-vif-deleted:0ae0efa3-a3a2-4fa2-957d-d2b7715eef7a for instance bddb57e6-3dc8-4a9a-a551-4727ff237620 on hv-cinder-maste Jul 11 14:48:47.788990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-d24ccdae-bbb5-49ba-b7b9-0cfe3285286b tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:47.789260 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-882754b5-d0a8-41e6-b1f3-b9cf1e33998e req-18f3e563-f552-42e1-ba29-e51313bd334f service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047232 Jul 11 14:48:47.794777 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 508/1046] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:48:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:48:47.801831 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-d24ccdae-bbb5-49ba-b7b9-0cfe3285286b tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620" status: 200 len: 1505 microversion: 2.1 time: 0.423729 Jul 11 14:48:47.802780 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 539/1047] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:47 2022] GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620 => generated 1505 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:48.820486 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-55c1020c-e1cf-4104-bb12-ef6d70aff95b tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:48.822035 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-55c1020c-e1cf-4104-bb12-ef6d70aff95b tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: bddb57e6-3dc8-4a9a-a551-4727ff237620] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:48.829756 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-55c1020c-e1cf-4104-bb12-ef6d70aff95b tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:48.830307 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-55c1020c-e1cf-4104-bb12-ef6d70aff95b tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:48.831008 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-55c1020c-e1cf-4104-bb12-ef6d70aff95b tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:48.831666 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-55c1020c-e1cf-4104-bb12-ef6d70aff95b tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:48.848475 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-55c1020c-e1cf-4104-bb12-ef6d70aff95b tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] HTTP exception thrown: Instance bddb57e6-3dc8-4a9a-a551-4727ff237620 could not be found. Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-55c1020c-e1cf-4104-bb12-ef6d70aff95b tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Request method failure captured: Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620 HTTP/1.1 Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDg990OU4G9CRNtAGmiufJcvz-dz0sveAedjQEBM7CRyv0yl8-vxMGt-wXyxINlakiy9btNKkHmvHVLkQF0jsLS_ViLVFz9F05OBJ6xGM-n6HmncsS7OlzG9IYStSCJY8DBzmv_Jqf47Bs-2qhSc6EHwD9QR--m9vxFJaB9Y3Va3ZFc Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: 23a54d0a3442474db1829cb42e2b3c95 Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-VolumesActionsTest-346082276 Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:48:48.850536 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:48:48.854367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/23a54d0a3442474db1829cb42e2b3c95"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/23a54d0a3442474db1829cb42e2b3c95"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_23a54d0a3442474db1829cb42e2b3c95"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/23a54d0a3442474db1829cb42e2b3c95"}]}] Jul 11 14:48:48.854367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-VolumesActionsTest-346082276 Jul 11 14:48:48.854367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: 23a54d0a3442474db1829cb42e2b3c95 Jul 11 14:48:48.854367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-VolumesActionsTest-346082276 Jul 11 14:48:48.854367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-VolumesActionsTest-346082276-project Jul 11 14:48:48.854367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:48:48.854367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:48:48.854367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 6e9ce7b0fdbd43f28d38a4b5657a3e2e Jul 11 14:48:48.854367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-VolumesActionsTest-346082276-project Jul 11 14:48:48.854367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:48:48.854367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance bddb57e6-3dc8-4a9a-a551-4727ff237620 could not be found. Jul 11 14:48:48.854367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': 'bddb57e6-3dc8-4a9a-a551-4727ff237620'} Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bddb57e6-3dc8-4a9a-a551-4727ff237620 could not be found. Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:48:48.857665 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:48:48.863736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:48:48.863736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:48:48.863736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:48:48.863736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:48:48.863736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:48:48.863736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:48:48.863736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:48:48.863736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bddb57e6-3dc8-4a9a-a551-4727ff237620 could not be found. Jul 11 14:48:48.863736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:48.863736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:48:48.863736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:48.863736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:48:48.863736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:48:48.863736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:48:48.863736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:48:48.863736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:48:48.863736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bddb57e6-3dc8-4a9a-a551-4727ff237620 could not be found. Jul 11 14:48:48.863736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:48:48.866926 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-55c1020c-e1cf-4104-bb12-ef6d70aff95b tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Returning 404 to user: Instance bddb57e6-3dc8-4a9a-a551-4727ff237620 could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:48:48.866926 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-55c1020c-e1cf-4104-bb12-ef6d70aff95b tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620" status: 404 len: 111 microversion: 2.1 time: 0.036409 Jul 11 14:48:48.866926 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 509/1048] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:48 2022] GET /compute/v2.1/servers/bddb57e6-3dc8-4a9a-a551-4727ff237620 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:48:48.979380 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-28ce909f-d541-4ae4-8532-ad73a0182f8a tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-783171062", "imageRef": "a588fa24-23b5-4e53-87c6-7e600ee3a5ef", "flavorRef": "452", "networks": [{"uuid": "558e7e14-28fa-468b-8d85-62c3486f900d"}]}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:48:49.054025 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.network.neutron [None req-28ce909f-d541-4ae4-8532-ad73a0182f8a tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] validate_networks() for [('558e7e14-28fa-468b-8d85-62c3486f900d', None, None, None, None, None)] {{(pid=130159) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:48:49.652911 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-28ce909f-d541-4ae4-8532-ad73a0182f8a tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Flavor limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:48:49.653754 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-28ce909f-d541-4ae4-8532-ad73a0182f8a tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Image limits 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:48:49.654855 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-28ce909f-d541-4ae4-8532-ad73a0182f8a tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Flavor pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:48:49.655649 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-28ce909f-d541-4ae4-8532-ad73a0182f8a tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Image pref 0:0:0 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:48:49.656455 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.virt.hardware [None req-28ce909f-d541-4ae4-8532-ad73a0182f8a tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130159) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:48:49.677868 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-28ce909f-d541-4ae4-8532-ad73a0182f8a tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:49.678743 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-28ce909f-d541-4ae4-8532-ad73a0182f8a tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:49.680400 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-28ce909f-d541-4ae4-8532-ad73a0182f8a tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:49.681220 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-28ce909f-d541-4ae4-8532-ad73a0182f8a tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:49.720145 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-28ce909f-d541-4ae4-8532-ad73a0182f8a tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Getting quotas for project 23a54d0a3442474db1829cb42e2b3c95. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:48:49.728588 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.quota [None req-28ce909f-d541-4ae4-8532-ad73a0182f8a tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Getting quotas for user 6e9ce7b0fdbd43f28d38a4b5657a3e2e and project 23a54d0a3442474db1829cb42e2b3c95. Resources: {'ram', 'cores', 'instances'} {{(pid=130159) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:48:49.753276 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-28ce909f-d541-4ae4-8532-ad73a0182f8a tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Going to run 1 instances... {{(pid=130159) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:48:49.757951 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-28ce909f-d541-4ae4-8532-ad73a0182f8a tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: 20001341-a787-44a7-9010-f8c8abd34313] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a588fa24-23b5-4e53-87c6-7e600ee3a5ef',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130159) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:48:49.849516 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-28ce909f-d541-4ae4-8532-ad73a0182f8a tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.874825 Jul 11 14:48:49.859499 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 540/1049] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:48:48 2022] POST /compute/v2.1/servers => generated 380 bytes in 886 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Jul 11 14:48:49.872974 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8015d48f-aef2-4dfe-9804-6d1624151475 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:49.875390 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-8015d48f-aef2-4dfe-9804-6d1624151475 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: 20001341-a787-44a7-9010-f8c8abd34313] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:49.977278 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8015d48f-aef2-4dfe-9804-6d1624151475 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:49.992652 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8015d48f-aef2-4dfe-9804-6d1624151475 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313" status: 200 len: 1213 microversion: 2.1 time: 0.130586 Jul 11 14:48:49.993902 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 510/1050] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:49 2022] GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313 => generated 1213 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:51.019887 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e2f46f45-4842-4253-8f84-a95f8e3d6564 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:51.021959 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e2f46f45-4842-4253-8f84-a95f8e3d6564 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: 20001341-a787-44a7-9010-f8c8abd34313] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:51.033935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e2f46f45-4842-4253-8f84-a95f8e3d6564 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:51.035209 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e2f46f45-4842-4253-8f84-a95f8e3d6564 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:51.036660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e2f46f45-4842-4253-8f84-a95f8e3d6564 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:51.037419 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-e2f46f45-4842-4253-8f84-a95f8e3d6564 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:51.181525 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e2f46f45-4842-4253-8f84-a95f8e3d6564 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:51.202234 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e2f46f45-4842-4253-8f84-a95f8e3d6564 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313" status: 200 len: 1265 microversion: 2.1 time: 0.190182 Jul 11 14:48:51.203592 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 541/1051] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:51 2022] GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313 => generated 1265 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:52.230823 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-62c1b6b6-fc9b-435d-b54f-6cfd5922241e tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:52.231680 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-62c1b6b6-fc9b-435d-b54f-6cfd5922241e tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: 20001341-a787-44a7-9010-f8c8abd34313] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:52.241004 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-62c1b6b6-fc9b-435d-b54f-6cfd5922241e tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:52.241664 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-62c1b6b6-fc9b-435d-b54f-6cfd5922241e tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:52.242721 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-62c1b6b6-fc9b-435d-b54f-6cfd5922241e tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:52.243161 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-62c1b6b6-fc9b-435d-b54f-6cfd5922241e tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:52.574653 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-62c1b6b6-fc9b-435d-b54f-6cfd5922241e tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:52.587443 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-62c1b6b6-fc9b-435d-b54f-6cfd5922241e tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313" status: 200 len: 1313 microversion: 2.1 time: 0.364311 Jul 11 14:48:52.589086 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 511/1052] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:52 2022] GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313 => generated 1313 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:53.188159 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-4a9449ca-edfd-45a1-b579-babb967ab98f req-91134905-e396-405f-b225-134d52041ad4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "20001341-a787-44a7-9010-f8c8abd34313", "tag": "7a3699df-313a-417b-bb80-b9981d10da5d"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:48:53.212473 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-4a9449ca-edfd-45a1-b579-babb967ab98f req-91134905-e396-405f-b225-134d52041ad4 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:53.214144 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-4a9449ca-edfd-45a1-b579-babb967ab98f req-91134905-e396-405f-b225-134d52041ad4 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:53.243256 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-4a9449ca-edfd-45a1-b579-babb967ab98f req-91134905-e396-405f-b225-134d52041ad4 service nova] Creating event network-changed:7a3699df-313a-417b-bb80-b9981d10da5d for instance 20001341-a787-44a7-9010-f8c8abd34313 on hv-cinder-maste Jul 11 14:48:53.250327 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-4a9449ca-edfd-45a1-b579-babb967ab98f req-91134905-e396-405f-b225-134d52041ad4 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072493 Jul 11 14:48:53.252343 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 542/1053] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:48:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:48:53.618868 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f9fe094b-9ad1-45fe-bfc1-d96d5682a199 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:53.620957 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f9fe094b-9ad1-45fe-bfc1-d96d5682a199 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: 20001341-a787-44a7-9010-f8c8abd34313] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:53.632849 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f9fe094b-9ad1-45fe-bfc1-d96d5682a199 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:53.633671 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f9fe094b-9ad1-45fe-bfc1-d96d5682a199 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:53.635071 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f9fe094b-9ad1-45fe-bfc1-d96d5682a199 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:53.635978 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f9fe094b-9ad1-45fe-bfc1-d96d5682a199 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:53.907074 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f9fe094b-9ad1-45fe-bfc1-d96d5682a199 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:53.918610 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f9fe094b-9ad1-45fe-bfc1-d96d5682a199 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313" status: 200 len: 1313 microversion: 2.1 time: 0.307317 Jul 11 14:48:53.926293 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 512/1054] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:53 2022] GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313 => generated 1313 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:54.947490 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-311d021e-af38-4ceb-9866-33fe32267c5c tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:54.948787 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-311d021e-af38-4ceb-9866-33fe32267c5c tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: 20001341-a787-44a7-9010-f8c8abd34313] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:54.960843 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-311d021e-af38-4ceb-9866-33fe32267c5c tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:54.961487 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-311d021e-af38-4ceb-9866-33fe32267c5c tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:54.962746 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-311d021e-af38-4ceb-9866-33fe32267c5c tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:54.963456 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-311d021e-af38-4ceb-9866-33fe32267c5c tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:55.177893 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-311d021e-af38-4ceb-9866-33fe32267c5c tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:55.198718 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-311d021e-af38-4ceb-9866-33fe32267c5c tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313" status: 200 len: 1313 microversion: 2.1 time: 0.263482 Jul 11 14:48:55.200400 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 543/1055] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:54 2022] GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313 => generated 1313 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:56.219286 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-8310a85d-2c83-4256-9912-7447e0eec0a7 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:56.220245 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-8310a85d-2c83-4256-9912-7447e0eec0a7 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: 20001341-a787-44a7-9010-f8c8abd34313] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:56.228286 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8310a85d-2c83-4256-9912-7447e0eec0a7 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:56.228835 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8310a85d-2c83-4256-9912-7447e0eec0a7 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:56.230215 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8310a85d-2c83-4256-9912-7447e0eec0a7 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:56.230809 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-8310a85d-2c83-4256-9912-7447e0eec0a7 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:56.465018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-8310a85d-2c83-4256-9912-7447e0eec0a7 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:56.475270 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-8310a85d-2c83-4256-9912-7447e0eec0a7 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313" status: 200 len: 1313 microversion: 2.1 time: 0.261180 Jul 11 14:48:56.475861 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 513/1056] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:56 2022] GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313 => generated 1313 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:57.504660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-86831297-64c9-4546-8a05-f2f1bd97dbc4 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:57.507018 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-86831297-64c9-4546-8a05-f2f1bd97dbc4 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: 20001341-a787-44a7-9010-f8c8abd34313] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:57.521023 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-86831297-64c9-4546-8a05-f2f1bd97dbc4 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:57.522510 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-86831297-64c9-4546-8a05-f2f1bd97dbc4 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:57.524183 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-86831297-64c9-4546-8a05-f2f1bd97dbc4 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:57.525499 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-86831297-64c9-4546-8a05-f2f1bd97dbc4 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:57.839570 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-86831297-64c9-4546-8a05-f2f1bd97dbc4 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:57.856652 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-86831297-64c9-4546-8a05-f2f1bd97dbc4 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313" status: 200 len: 1313 microversion: 2.1 time: 0.360672 Jul 11 14:48:57.865313 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 544/1057] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:57 2022] GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313 => generated 1313 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:48:58.880415 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-be43c514-0183-43f5-a947-2a65a43d7fcf tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:48:58.881136 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-be43c514-0183-43f5-a947-2a65a43d7fcf tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: 20001341-a787-44a7-9010-f8c8abd34313] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:48:58.888730 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-be43c514-0183-43f5-a947-2a65a43d7fcf tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:58.890017 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-be43c514-0183-43f5-a947-2a65a43d7fcf tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:58.891105 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-be43c514-0183-43f5-a947-2a65a43d7fcf tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:48:58.891766 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-be43c514-0183-43f5-a947-2a65a43d7fcf tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:48:59.236154 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-be43c514-0183-43f5-a947-2a65a43d7fcf tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:48:59.253415 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-be43c514-0183-43f5-a947-2a65a43d7fcf tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313" status: 200 len: 1313 microversion: 2.1 time: 0.376602 Jul 11 14:48:59.254962 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 514/1058] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:48:58 2022] GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313 => generated 1313 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:49:00.091051 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-e1492c30-816e-4b15-bf35-729cc5c3ea1b req-fe629163-d0e2-49ce-9c1b-0398f9fda9ce service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "20001341-a787-44a7-9010-f8c8abd34313", "name": "network-vif-plugged", "status": "completed", "tag": "7a3699df-313a-417b-bb80-b9981d10da5d"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:49:00.097732 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=2 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:49:00.099481 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=6 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:49:00.110816 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-e1492c30-816e-4b15-bf35-729cc5c3ea1b req-fe629163-d0e2-49ce-9c1b-0398f9fda9ce service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:00.111643 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-e1492c30-816e-4b15-bf35-729cc5c3ea1b req-fe629163-d0e2-49ce-9c1b-0398f9fda9ce service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:00.138998 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-e1492c30-816e-4b15-bf35-729cc5c3ea1b req-fe629163-d0e2-49ce-9c1b-0398f9fda9ce service nova] Creating event network-vif-plugged:7a3699df-313a-417b-bb80-b9981d10da5d for instance 20001341-a787-44a7-9010-f8c8abd34313 on hv-cinder-maste Jul 11 14:49:00.147922 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-e1492c30-816e-4b15-bf35-729cc5c3ea1b req-fe629163-d0e2-49ce-9c1b-0398f9fda9ce service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.061867 Jul 11 14:49:00.148898 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 545/1059] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:49:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:49:00.279279 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f330de28-b88a-443e-b324-3348b0a0ec74 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:00.281415 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f330de28-b88a-443e-b324-3348b0a0ec74 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: 20001341-a787-44a7-9010-f8c8abd34313] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:00.286324 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=2 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:49:00.301805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f330de28-b88a-443e-b324-3348b0a0ec74 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:00.302673 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f330de28-b88a-443e-b324-3348b0a0ec74 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:00.304522 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f330de28-b88a-443e-b324-3348b0a0ec74 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:00.305270 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f330de28-b88a-443e-b324-3348b0a0ec74 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:00.570450 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f330de28-b88a-443e-b324-3348b0a0ec74 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:49:00.582452 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f330de28-b88a-443e-b324-3348b0a0ec74 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313" status: 200 len: 1313 microversion: 2.1 time: 0.307068 Jul 11 14:49:00.583540 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 515/1060] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:00 2022] GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313 => generated 1313 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:49:01.607677 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c229c074-505f-453f-88b2-e36effa367fc tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:01.609665 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c229c074-505f-453f-88b2-e36effa367fc tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: 20001341-a787-44a7-9010-f8c8abd34313] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:01.623461 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c229c074-505f-453f-88b2-e36effa367fc tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:01.624798 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c229c074-505f-453f-88b2-e36effa367fc tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:01.626793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c229c074-505f-453f-88b2-e36effa367fc tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:01.627969 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c229c074-505f-453f-88b2-e36effa367fc tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:01.860252 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c229c074-505f-453f-88b2-e36effa367fc tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:49:01.873214 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c229c074-505f-453f-88b2-e36effa367fc tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313" status: 200 len: 1313 microversion: 2.1 time: 0.274425 Jul 11 14:49:01.874701 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 546/1061] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:01 2022] GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313 => generated 1313 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:49:02.899775 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a7901dbe-bdb8-47bf-b1fc-7aec4f3028eb tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:02.900935 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a7901dbe-bdb8-47bf-b1fc-7aec4f3028eb tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: 20001341-a787-44a7-9010-f8c8abd34313] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:02.911701 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a7901dbe-bdb8-47bf-b1fc-7aec4f3028eb tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:02.912328 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a7901dbe-bdb8-47bf-b1fc-7aec4f3028eb tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:02.913244 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a7901dbe-bdb8-47bf-b1fc-7aec4f3028eb tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:02.914030 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a7901dbe-bdb8-47bf-b1fc-7aec4f3028eb tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:03.075792 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a7901dbe-bdb8-47bf-b1fc-7aec4f3028eb tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:49:03.085912 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a7901dbe-bdb8-47bf-b1fc-7aec4f3028eb tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313" status: 200 len: 1497 microversion: 2.1 time: 0.190188 Jul 11 14:49:03.086538 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 516/1062] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:02 2022] GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313 => generated 1497 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:49:03.727343 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-eab02a45-fc38-49ea-8e8d-03489fa35856 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:03.728222 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-eab02a45-fc38-49ea-8e8d-03489fa35856 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: 20001341-a787-44a7-9010-f8c8abd34313] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:03.737599 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-eab02a45-fc38-49ea-8e8d-03489fa35856 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:03.738235 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-eab02a45-fc38-49ea-8e8d-03489fa35856 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:03.739174 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-eab02a45-fc38-49ea-8e8d-03489fa35856 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:03.739793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-eab02a45-fc38-49ea-8e8d-03489fa35856 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:03.773417 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-eab02a45-fc38-49ea-8e8d-03489fa35856 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: 20001341-a787-44a7-9010-f8c8abd34313] Going to try to terminate instance {{(pid=130159) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:49:03.903107 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-eab02a45-fc38-49ea-8e8d-03489fa35856 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "DELETE /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313" status: 204 len: 0 microversion: 2.1 time: 0.179908 Jul 11 14:49:03.903706 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 547/1063] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:49:03 2022] DELETE /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Jul 11 14:49:03.919729 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-afb4dfd5-90ae-4325-a871-1495f89a0c03 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:03.921543 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-afb4dfd5-90ae-4325-a871-1495f89a0c03 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: 20001341-a787-44a7-9010-f8c8abd34313] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:03.934460 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-afb4dfd5-90ae-4325-a871-1495f89a0c03 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:03.935294 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-afb4dfd5-90ae-4325-a871-1495f89a0c03 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:03.937165 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-afb4dfd5-90ae-4325-a871-1495f89a0c03 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:03.937973 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-afb4dfd5-90ae-4325-a871-1495f89a0c03 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:04.135740 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-afb4dfd5-90ae-4325-a871-1495f89a0c03 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:49:04.145112 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-afb4dfd5-90ae-4325-a871-1495f89a0c03 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313" status: 200 len: 1503 microversion: 2.1 time: 0.235753 Jul 11 14:49:04.145892 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 517/1064] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:03 2022] GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313 => generated 1503 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:49:05.167980 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a5aa7ebf-e9b1-4e09-89a6-9ae81c51ee31 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:05.169076 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a5aa7ebf-e9b1-4e09-89a6-9ae81c51ee31 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: 20001341-a787-44a7-9010-f8c8abd34313] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:05.180025 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a5aa7ebf-e9b1-4e09-89a6-9ae81c51ee31 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:05.180758 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a5aa7ebf-e9b1-4e09-89a6-9ae81c51ee31 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:05.182019 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a5aa7ebf-e9b1-4e09-89a6-9ae81c51ee31 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:05.182789 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a5aa7ebf-e9b1-4e09-89a6-9ae81c51ee31 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:05.409639 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a5aa7ebf-e9b1-4e09-89a6-9ae81c51ee31 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '6e9ce7b0fdbd43f28d38a4b5657a3e2e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23a54d0a3442474db1829cb42e2b3c95', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:49:05.432924 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a5aa7ebf-e9b1-4e09-89a6-9ae81c51ee31 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313" status: 200 len: 1503 microversion: 2.1 time: 0.272809 Jul 11 14:49:05.434404 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 548/1065] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:05 2022] GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313 => generated 1503 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:49:05.910585 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-abdf10f6-cb28-481d-bbcd-8025666f3e44 req-f683f9d9-0ca3-4032-b580-74bd02890b7b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "20001341-a787-44a7-9010-f8c8abd34313", "name": "network-vif-deleted", "tag": "7a3699df-313a-417b-bb80-b9981d10da5d"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:49:05.926146 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-abdf10f6-cb28-481d-bbcd-8025666f3e44 req-f683f9d9-0ca3-4032-b580-74bd02890b7b service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:05.926759 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-abdf10f6-cb28-481d-bbcd-8025666f3e44 req-f683f9d9-0ca3-4032-b580-74bd02890b7b service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:05.940798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-abdf10f6-cb28-481d-bbcd-8025666f3e44 req-f683f9d9-0ca3-4032-b580-74bd02890b7b service nova] Creating event network-vif-deleted:7a3699df-313a-417b-bb80-b9981d10da5d for instance 20001341-a787-44a7-9010-f8c8abd34313 on hv-cinder-maste Jul 11 14:49:05.948200 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-abdf10f6-cb28-481d-bbcd-8025666f3e44 req-f683f9d9-0ca3-4032-b580-74bd02890b7b service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041377 Jul 11 14:49:05.948809 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 518/1066] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:49:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:49:06.468860 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2fccfa21-a3cd-4037-80f8-134e38b55434 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:06.469765 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-2fccfa21-a3cd-4037-80f8-134e38b55434 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] [instance: 20001341-a787-44a7-9010-f8c8abd34313] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:06.481664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2fccfa21-a3cd-4037-80f8-134e38b55434 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:06.482622 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2fccfa21-a3cd-4037-80f8-134e38b55434 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:06.484100 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2fccfa21-a3cd-4037-80f8-134e38b55434 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:06.484875 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-2fccfa21-a3cd-4037-80f8-134e38b55434 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:06.511877 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-2fccfa21-a3cd-4037-80f8-134e38b55434 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] HTTP exception thrown: Instance 20001341-a787-44a7-9010-f8c8abd34313 could not be found. Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2fccfa21-a3cd-4037-80f8-134e38b55434 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Request method failure captured: Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313 HTTP/1.1 Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDg990OU4G9CRNtAGmiufJcvz-dz0sveAedjQEBM7CRyv0yl8-vxMGt-wXyxINlakiy9btNKkHmvHVLkQF0jsLS_ViLVFz9F05OBJ6xGM-n6HmncsS7OlzG9IYStSCJY8DBzmv_Jqf47Bs-2qhSc6EHwD9QR--m9vxFJaB9Y3Va3ZFc Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: 23a54d0a3442474db1829cb42e2b3c95 Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-VolumesActionsTest-346082276 Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:49:06.516254 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:49:06.517990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/23a54d0a3442474db1829cb42e2b3c95"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/23a54d0a3442474db1829cb42e2b3c95"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_23a54d0a3442474db1829cb42e2b3c95"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/23a54d0a3442474db1829cb42e2b3c95"}]}] Jul 11 14:49:06.517990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-VolumesActionsTest-346082276 Jul 11 14:49:06.517990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: 23a54d0a3442474db1829cb42e2b3c95 Jul 11 14:49:06.517990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-VolumesActionsTest-346082276 Jul 11 14:49:06.517990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-VolumesActionsTest-346082276-project Jul 11 14:49:06.517990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:49:06.517990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:49:06.517990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 6e9ce7b0fdbd43f28d38a4b5657a3e2e Jul 11 14:49:06.517990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-VolumesActionsTest-346082276-project Jul 11 14:49:06.517990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:49:06.517990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance 20001341-a787-44a7-9010-f8c8abd34313 could not be found. Jul 11 14:49:06.517990 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '20001341-a787-44a7-9010-f8c8abd34313'} Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 20001341-a787-44a7-9010-f8c8abd34313 could not be found. Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:49:06.518963 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:49:06.520537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:49:06.520537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:49:06.520537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:49:06.520537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:49:06.520537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:49:06.520537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:49:06.520537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:49:06.520537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 20001341-a787-44a7-9010-f8c8abd34313 could not be found. Jul 11 14:49:06.520537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:49:06.520537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:49:06.520537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:49:06.520537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:49:06.520537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:49:06.520537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:49:06.520537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:49:06.520537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:49:06.520537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 20001341-a787-44a7-9010-f8c8abd34313 could not be found. Jul 11 14:49:06.520537 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:49:06.521795 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-2fccfa21-a3cd-4037-80f8-134e38b55434 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] Returning 404 to user: Instance 20001341-a787-44a7-9010-f8c8abd34313 could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:49:06.521795 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-2fccfa21-a3cd-4037-80f8-134e38b55434 tempest-VolumesActionsTest-346082276 tempest-VolumesActionsTest-346082276-project] 10.222.0.72 "GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313" status: 404 len: 111 microversion: 2.1 time: 0.054676 Jul 11 14:49:06.521795 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 549/1067] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:06 2022] GET /compute/v2.1/servers/20001341-a787-44a7-9010-f8c8abd34313 => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 11 14:49:35.702016 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9031a39f-c959-4035-bc2a-a002d55a400e tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-972220474", "imageRef": "a588fa24-23b5-4e53-87c6-7e600ee3a5ef", "flavorRef": "452"}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:49:35.708544 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=25 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:49:35.709199 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=53 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:49:35.760646 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.network.neutron [None req-9031a39f-c959-4035-bc2a-a002d55a400e tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] validate_networks() for None {{(pid=130160) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:49:36.074237 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-9031a39f-c959-4035-bc2a-a002d55a400e tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Flavor limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:49:36.074717 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-9031a39f-c959-4035-bc2a-a002d55a400e tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Image limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:49:36.075189 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-9031a39f-c959-4035-bc2a-a002d55a400e tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Flavor pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:49:36.075591 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-9031a39f-c959-4035-bc2a-a002d55a400e tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Image pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:49:36.076002 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-9031a39f-c959-4035-bc2a-a002d55a400e tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:49:36.087249 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9031a39f-c959-4035-bc2a-a002d55a400e tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:36.087697 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9031a39f-c959-4035-bc2a-a002d55a400e tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:36.088492 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9031a39f-c959-4035-bc2a-a002d55a400e tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:36.088927 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9031a39f-c959-4035-bc2a-a002d55a400e tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:36.108186 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-9031a39f-c959-4035-bc2a-a002d55a400e tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Getting quotas for project c467ecb2524b41e4840d81dd2128396a. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:49:36.113619 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-9031a39f-c959-4035-bc2a-a002d55a400e tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Getting quotas for user 47a0b93509ee497e98c3db9947c16e80 and project c467ecb2524b41e4840d81dd2128396a. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:49:36.128235 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-9031a39f-c959-4035-bc2a-a002d55a400e tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Going to run 1 instances... {{(pid=130160) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:49:36.130755 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-9031a39f-c959-4035-bc2a-a002d55a400e tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] [instance: 4e5e7ae8-7506-45ec-89eb-3e0a454e370d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,guest_format=None,id=,image_id='a588fa24-23b5-4e53-87c6-7e600ee3a5ef',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=130160) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:49:36.183278 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9031a39f-c959-4035-bc2a-a002d55a400e tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.485484 Jul 11 14:49:36.184049 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 519/1068] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:49:35 2022] POST /compute/v2.1/servers => generated 380 bytes in 487 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 11 14:49:36.196720 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-e0fdbb5b-e425-4e7a-9629-541b58530349 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:36.197800 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-e0fdbb5b-e425-4e7a-9629-541b58530349 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] [instance: 4e5e7ae8-7506-45ec-89eb-3e0a454e370d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:36.200464 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=37,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:49:36.201696 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=63,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:49:36.295853 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-e0fdbb5b-e425-4e7a-9629-541b58530349 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47a0b93509ee497e98c3db9947c16e80', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c467ecb2524b41e4840d81dd2128396a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:49:36.304479 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-e0fdbb5b-e425-4e7a-9629-541b58530349 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] 10.222.0.72 "GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d" status: 200 len: 1214 microversion: 2.1 time: 0.112640 Jul 11 14:49:36.305255 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 550/1069] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:36 2022] GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d => generated 1214 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:49:37.328850 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3d2a6523-14fd-4661-839b-a119ee47a0c2 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:37.330384 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-3d2a6523-14fd-4661-839b-a119ee47a0c2 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] [instance: 4e5e7ae8-7506-45ec-89eb-3e0a454e370d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:37.340028 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d2a6523-14fd-4661-839b-a119ee47a0c2 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:37.340521 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d2a6523-14fd-4661-839b-a119ee47a0c2 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:37.341983 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d2a6523-14fd-4661-839b-a119ee47a0c2 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:37.342890 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d2a6523-14fd-4661-839b-a119ee47a0c2 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:37.507858 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-3d2a6523-14fd-4661-839b-a119ee47a0c2 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47a0b93509ee497e98c3db9947c16e80', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c467ecb2524b41e4840d81dd2128396a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:49:37.526591 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-3d2a6523-14fd-4661-839b-a119ee47a0c2 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] 10.222.0.72 "GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d" status: 200 len: 1274 microversion: 2.1 time: 0.205025 Jul 11 14:49:37.531197 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 520/1070] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:37 2022] GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d => generated 1274 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:49:38.546389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-b99a064d-9689-4051-b634-99bef8896a5b tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:38.547299 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-b99a064d-9689-4051-b634-99bef8896a5b tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] [instance: 4e5e7ae8-7506-45ec-89eb-3e0a454e370d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:38.555771 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b99a064d-9689-4051-b634-99bef8896a5b tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:38.556631 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b99a064d-9689-4051-b634-99bef8896a5b tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:38.557655 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b99a064d-9689-4051-b634-99bef8896a5b tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:38.558203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-b99a064d-9689-4051-b634-99bef8896a5b tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:38.784203 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-b99a064d-9689-4051-b634-99bef8896a5b tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47a0b93509ee497e98c3db9947c16e80', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c467ecb2524b41e4840d81dd2128396a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:49:38.797778 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-b99a064d-9689-4051-b634-99bef8896a5b tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] 10.222.0.72 "GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d" status: 200 len: 1314 microversion: 2.1 time: 0.256429 Jul 11 14:49:38.799464 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 551/1071] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:38 2022] GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d => generated 1314 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:49:39.735693 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-fbc24bd7-5981-48f3-9092-058407a1991f req-1afa4dfc-348b-4dd2-9803-50a3cd872a8c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4e5e7ae8-7506-45ec-89eb-3e0a454e370d", "tag": "7bc14979-1242-4a53-af40-30c73f0c39a3"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:49:39.758873 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-fbc24bd7-5981-48f3-9092-058407a1991f req-1afa4dfc-348b-4dd2-9803-50a3cd872a8c service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:39.760593 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-fbc24bd7-5981-48f3-9092-058407a1991f req-1afa4dfc-348b-4dd2-9803-50a3cd872a8c service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:39.788435 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-fbc24bd7-5981-48f3-9092-058407a1991f req-1afa4dfc-348b-4dd2-9803-50a3cd872a8c service nova] Creating event network-changed:7bc14979-1242-4a53-af40-30c73f0c39a3 for instance 4e5e7ae8-7506-45ec-89eb-3e0a454e370d on hv-cinder-maste Jul 11 14:49:39.797029 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-fbc24bd7-5981-48f3-9092-058407a1991f req-1afa4dfc-348b-4dd2-9803-50a3cd872a8c service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068607 Jul 11 14:49:39.799067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 521/1072] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:49:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:49:39.821824 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-90bb3184-98ae-47f2-a9a6-98cd1442c676 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:39.823014 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-90bb3184-98ae-47f2-a9a6-98cd1442c676 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] [instance: 4e5e7ae8-7506-45ec-89eb-3e0a454e370d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:39.836200 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-90bb3184-98ae-47f2-a9a6-98cd1442c676 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:39.836968 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-90bb3184-98ae-47f2-a9a6-98cd1442c676 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:39.838378 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-90bb3184-98ae-47f2-a9a6-98cd1442c676 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:39.839174 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-90bb3184-98ae-47f2-a9a6-98cd1442c676 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:40.042883 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-90bb3184-98ae-47f2-a9a6-98cd1442c676 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47a0b93509ee497e98c3db9947c16e80', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c467ecb2524b41e4840d81dd2128396a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:49:40.055553 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-90bb3184-98ae-47f2-a9a6-98cd1442c676 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] 10.222.0.72 "GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d" status: 200 len: 1314 microversion: 2.1 time: 0.241002 Jul 11 14:49:40.056845 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 552/1073] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:39 2022] GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d => generated 1314 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:49:41.076462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-503b6752-7894-4342-89a9-044a8bcea250 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:41.077171 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-503b6752-7894-4342-89a9-044a8bcea250 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] [instance: 4e5e7ae8-7506-45ec-89eb-3e0a454e370d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:41.084629 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-503b6752-7894-4342-89a9-044a8bcea250 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:41.085081 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-503b6752-7894-4342-89a9-044a8bcea250 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:41.085883 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-503b6752-7894-4342-89a9-044a8bcea250 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:41.086321 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-503b6752-7894-4342-89a9-044a8bcea250 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:41.331891 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-503b6752-7894-4342-89a9-044a8bcea250 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47a0b93509ee497e98c3db9947c16e80', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c467ecb2524b41e4840d81dd2128396a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:49:41.340326 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-503b6752-7894-4342-89a9-044a8bcea250 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] 10.222.0.72 "GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d" status: 200 len: 1314 microversion: 2.1 time: 0.268567 Jul 11 14:49:41.340912 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 522/1074] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:41 2022] GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d => generated 1314 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:49:42.358342 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-eee44ca7-f2f6-4f03-9372-f8a07a0fb8df tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:42.359257 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-eee44ca7-f2f6-4f03-9372-f8a07a0fb8df tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] [instance: 4e5e7ae8-7506-45ec-89eb-3e0a454e370d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:42.367041 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-eee44ca7-f2f6-4f03-9372-f8a07a0fb8df tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:42.367564 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-eee44ca7-f2f6-4f03-9372-f8a07a0fb8df tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:42.368482 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-eee44ca7-f2f6-4f03-9372-f8a07a0fb8df tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:42.368968 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-eee44ca7-f2f6-4f03-9372-f8a07a0fb8df tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:42.521364 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-eee44ca7-f2f6-4f03-9372-f8a07a0fb8df tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47a0b93509ee497e98c3db9947c16e80', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c467ecb2524b41e4840d81dd2128396a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:49:42.529506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-eee44ca7-f2f6-4f03-9372-f8a07a0fb8df tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] 10.222.0.72 "GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d" status: 200 len: 1314 microversion: 2.1 time: 0.175252 Jul 11 14:49:42.530103 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 553/1075] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:42 2022] GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d => generated 1314 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:49:43.548022 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-0bd6bd6f-4e85-4eea-b8d5-47d4b4ecceed tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:43.548699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-0bd6bd6f-4e85-4eea-b8d5-47d4b4ecceed tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] [instance: 4e5e7ae8-7506-45ec-89eb-3e0a454e370d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:43.555968 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0bd6bd6f-4e85-4eea-b8d5-47d4b4ecceed tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:43.556421 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0bd6bd6f-4e85-4eea-b8d5-47d4b4ecceed tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:43.557229 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0bd6bd6f-4e85-4eea-b8d5-47d4b4ecceed tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:43.557673 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0bd6bd6f-4e85-4eea-b8d5-47d4b4ecceed tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:43.753621 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-0bd6bd6f-4e85-4eea-b8d5-47d4b4ecceed tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47a0b93509ee497e98c3db9947c16e80', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c467ecb2524b41e4840d81dd2128396a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:49:43.762318 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-0bd6bd6f-4e85-4eea-b8d5-47d4b4ecceed tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] 10.222.0.72 "GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d" status: 200 len: 1314 microversion: 2.1 time: 0.218997 Jul 11 14:49:43.763363 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 523/1076] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:43 2022] GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d => generated 1314 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:49:44.785556 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-7b323352-2023-4325-b784-5c585f5fe002 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:44.786513 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-7b323352-2023-4325-b784-5c585f5fe002 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] [instance: 4e5e7ae8-7506-45ec-89eb-3e0a454e370d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:44.794084 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7b323352-2023-4325-b784-5c585f5fe002 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:44.795428 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7b323352-2023-4325-b784-5c585f5fe002 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:44.796239 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7b323352-2023-4325-b784-5c585f5fe002 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:44.796748 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7b323352-2023-4325-b784-5c585f5fe002 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:44.974685 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7b323352-2023-4325-b784-5c585f5fe002 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47a0b93509ee497e98c3db9947c16e80', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c467ecb2524b41e4840d81dd2128396a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:49:44.984642 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-7b323352-2023-4325-b784-5c585f5fe002 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] 10.222.0.72 "GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d" status: 200 len: 1314 microversion: 2.1 time: 0.203317 Jul 11 14:49:44.985339 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 554/1077] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:44 2022] GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d => generated 1314 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:49:45.830976 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-8e11e642-5d5c-48e7-b912-6094d886b3bf req-dcca1a7b-20a6-4b76-8f67-719b05604e7d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4e5e7ae8-7506-45ec-89eb-3e0a454e370d", "name": "network-vif-plugged", "status": "completed", "tag": "7bc14979-1242-4a53-af40-30c73f0c39a3"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:49:45.842987 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-8e11e642-5d5c-48e7-b912-6094d886b3bf req-dcca1a7b-20a6-4b76-8f67-719b05604e7d service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:45.843531 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-8e11e642-5d5c-48e7-b912-6094d886b3bf req-dcca1a7b-20a6-4b76-8f67-719b05604e7d service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:45.856229 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-8e11e642-5d5c-48e7-b912-6094d886b3bf req-dcca1a7b-20a6-4b76-8f67-719b05604e7d service nova] Creating event network-vif-plugged:7bc14979-1242-4a53-af40-30c73f0c39a3 for instance 4e5e7ae8-7506-45ec-89eb-3e0a454e370d on hv-cinder-maste Jul 11 14:49:45.862086 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-8e11e642-5d5c-48e7-b912-6094d886b3bf req-dcca1a7b-20a6-4b76-8f67-719b05604e7d service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036015 Jul 11 14:49:45.863188 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 524/1078] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:49:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:49:46.003097 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a44ad8b6-b9c4-4ca1-92a2-c56860663eed tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:46.003886 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a44ad8b6-b9c4-4ca1-92a2-c56860663eed tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] [instance: 4e5e7ae8-7506-45ec-89eb-3e0a454e370d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:46.011595 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a44ad8b6-b9c4-4ca1-92a2-c56860663eed tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:46.012053 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a44ad8b6-b9c4-4ca1-92a2-c56860663eed tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:46.012864 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a44ad8b6-b9c4-4ca1-92a2-c56860663eed tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:46.013867 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a44ad8b6-b9c4-4ca1-92a2-c56860663eed tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:46.193666 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-a44ad8b6-b9c4-4ca1-92a2-c56860663eed tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47a0b93509ee497e98c3db9947c16e80', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c467ecb2524b41e4840d81dd2128396a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:49:46.202909 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a44ad8b6-b9c4-4ca1-92a2-c56860663eed tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] 10.222.0.72 "GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d" status: 200 len: 1314 microversion: 2.1 time: 0.205132 Jul 11 14:49:46.203510 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 555/1079] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:45 2022] GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d => generated 1314 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:49:47.218269 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=2 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:49:47.219176 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=6 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:49:47.225023 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-f919b080-71ea-40a6-a411-889d49ccaae1 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:47.225849 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-f919b080-71ea-40a6-a411-889d49ccaae1 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] [instance: 4e5e7ae8-7506-45ec-89eb-3e0a454e370d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:47.233789 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f919b080-71ea-40a6-a411-889d49ccaae1 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:47.234268 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f919b080-71ea-40a6-a411-889d49ccaae1 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:47.235050 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f919b080-71ea-40a6-a411-889d49ccaae1 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:47.235484 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-f919b080-71ea-40a6-a411-889d49ccaae1 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:47.418176 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-f919b080-71ea-40a6-a411-889d49ccaae1 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47a0b93509ee497e98c3db9947c16e80', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c467ecb2524b41e4840d81dd2128396a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:49:47.428362 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-f919b080-71ea-40a6-a411-889d49ccaae1 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] 10.222.0.72 "GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d" status: 200 len: 1314 microversion: 2.1 time: 0.210512 Jul 11 14:49:47.429029 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 525/1080] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:47 2022] GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d => generated 1314 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:49:48.447500 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f27e6e5e-2a1e-4185-a45b-dc06f4213dc6 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:48.448222 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f27e6e5e-2a1e-4185-a45b-dc06f4213dc6 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] [instance: 4e5e7ae8-7506-45ec-89eb-3e0a454e370d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:48.450383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=2 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:49:48.457947 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f27e6e5e-2a1e-4185-a45b-dc06f4213dc6 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:48.458697 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f27e6e5e-2a1e-4185-a45b-dc06f4213dc6 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:48.459631 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f27e6e5e-2a1e-4185-a45b-dc06f4213dc6 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:48.460131 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f27e6e5e-2a1e-4185-a45b-dc06f4213dc6 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:48.611114 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f27e6e5e-2a1e-4185-a45b-dc06f4213dc6 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47a0b93509ee497e98c3db9947c16e80', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c467ecb2524b41e4840d81dd2128396a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:49:48.619980 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f27e6e5e-2a1e-4185-a45b-dc06f4213dc6 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] 10.222.0.72 "GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d" status: 200 len: 1464 microversion: 2.1 time: 0.177275 Jul 11 14:49:48.620939 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 556/1081] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:48 2022] GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d => generated 1464 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:49:48.664339 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-0b76f120-e952-47da-815b-c54b5cd02cac tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:48.665172 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-0b76f120-e952-47da-815b-c54b5cd02cac tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] [instance: 4e5e7ae8-7506-45ec-89eb-3e0a454e370d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:48.672371 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0b76f120-e952-47da-815b-c54b5cd02cac tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:48.672974 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0b76f120-e952-47da-815b-c54b5cd02cac tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:48.673779 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0b76f120-e952-47da-815b-c54b5cd02cac tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:48.674249 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0b76f120-e952-47da-815b-c54b5cd02cac tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:48.702820 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-0b76f120-e952-47da-815b-c54b5cd02cac tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] [instance: 4e5e7ae8-7506-45ec-89eb-3e0a454e370d] Going to try to terminate instance {{(pid=130160) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:49:48.793343 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-0b76f120-e952-47da-815b-c54b5cd02cac tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] 10.222.0.72 "DELETE /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d" status: 204 len: 0 microversion: 2.1 time: 0.132511 Jul 11 14:49:48.794341 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 526/1082] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:49:48 2022] DELETE /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 11 14:49:48.804272 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-4cc411eb-c490-4a82-846a-f7bbf9f865a8 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:48.804935 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-4cc411eb-c490-4a82-846a-f7bbf9f865a8 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] [instance: 4e5e7ae8-7506-45ec-89eb-3e0a454e370d] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:48.811547 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4cc411eb-c490-4a82-846a-f7bbf9f865a8 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:48.812004 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4cc411eb-c490-4a82-846a-f7bbf9f865a8 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:48.812793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4cc411eb-c490-4a82-846a-f7bbf9f865a8 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:48.813241 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-4cc411eb-c490-4a82-846a-f7bbf9f865a8 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:48.981178 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-4cc411eb-c490-4a82-846a-f7bbf9f865a8 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47a0b93509ee497e98c3db9947c16e80', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c467ecb2524b41e4840d81dd2128396a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:49:48.994764 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-4cc411eb-c490-4a82-846a-f7bbf9f865a8 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] 10.222.0.72 "GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d" status: 200 len: 1470 microversion: 2.1 time: 0.194076 Jul 11 14:49:48.995836 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 557/1083] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:48 2022] GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d => generated 1470 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:49:50.016410 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-5feb936f-c953-4fea-bb18-c38d42bc1b82 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:50.017232 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-5feb936f-c953-4fea-bb18-c38d42bc1b82 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] [instance: 4e5e7ae8-7506-45ec-89eb-3e0a454e370d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:50.024797 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5feb936f-c953-4fea-bb18-c38d42bc1b82 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:50.025249 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5feb936f-c953-4fea-bb18-c38d42bc1b82 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:50.026038 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5feb936f-c953-4fea-bb18-c38d42bc1b82 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:50.026518 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-5feb936f-c953-4fea-bb18-c38d42bc1b82 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:50.213419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-5feb936f-c953-4fea-bb18-c38d42bc1b82 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '47a0b93509ee497e98c3db9947c16e80', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c467ecb2524b41e4840d81dd2128396a', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:49:50.223585 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-5feb936f-c953-4fea-bb18-c38d42bc1b82 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] 10.222.0.72 "GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d" status: 200 len: 1470 microversion: 2.1 time: 0.211840 Jul 11 14:49:50.224185 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 527/1084] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:50 2022] GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d => generated 1470 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:49:50.733811 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-d2238574-5e90-4c41-bb81-ef44476c478a req-651c0988-38a2-44cf-907a-5900825c2ca6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4e5e7ae8-7506-45ec-89eb-3e0a454e370d", "name": "network-vif-deleted", "tag": "7bc14979-1242-4a53-af40-30c73f0c39a3"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:49:50.756052 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-d2238574-5e90-4c41-bb81-ef44476c478a req-651c0988-38a2-44cf-907a-5900825c2ca6 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:50.756890 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-d2238574-5e90-4c41-bb81-ef44476c478a req-651c0988-38a2-44cf-907a-5900825c2ca6 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:50.777815 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-d2238574-5e90-4c41-bb81-ef44476c478a req-651c0988-38a2-44cf-907a-5900825c2ca6 service nova] Creating event network-vif-deleted:7bc14979-1242-4a53-af40-30c73f0c39a3 for instance 4e5e7ae8-7506-45ec-89eb-3e0a454e370d on hv-cinder-maste Jul 11 14:49:50.788225 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-d2238574-5e90-4c41-bb81-ef44476c478a req-651c0988-38a2-44cf-907a-5900825c2ca6 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.061703 Jul 11 14:49:50.789607 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 558/1085] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:49:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:49:51.245694 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-dd523309-03d2-4675-8b0f-489b7d61f011 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:49:51.246821 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-dd523309-03d2-4675-8b0f-489b7d61f011 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] [instance: 4e5e7ae8-7506-45ec-89eb-3e0a454e370d] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:49:51.259031 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dd523309-03d2-4675-8b0f-489b7d61f011 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:51.259843 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dd523309-03d2-4675-8b0f-489b7d61f011 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:51.261182 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dd523309-03d2-4675-8b0f-489b7d61f011 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:49:51.261958 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-dd523309-03d2-4675-8b0f-489b7d61f011 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:49:51.289523 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-dd523309-03d2-4675-8b0f-489b7d61f011 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] HTTP exception thrown: Instance 4e5e7ae8-7506-45ec-89eb-3e0a454e370d could not be found. Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-dd523309-03d2-4675-8b0f-489b7d61f011 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Request method failure captured: Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d HTTP/1.1 Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDh-uMgQTjo2X59g0zMqtlur0f-C3KK5zt4OBjFtGlk4id4MQBv_6hrwXUMEKwGgajr12wrTnq-gNI508G7pKYStKf88gLjiNOqtUmfpwsb7Amg0JiovRWIs7w__-Xor-Z958Mdg-H1PbUgqdnxAlUeEsRh2IhYMGT0Erbzilm-UDac Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: c467ecb2524b41e4840d81dd2128396a Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-VolumesNegativeTest-670728480 Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:49:51.292817 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:49:51.294725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/c467ecb2524b41e4840d81dd2128396a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/c467ecb2524b41e4840d81dd2128396a"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_c467ecb2524b41e4840d81dd2128396a"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/c467ecb2524b41e4840d81dd2128396a"}]}] Jul 11 14:49:51.294725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-VolumesNegativeTest-670728480 Jul 11 14:49:51.294725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: c467ecb2524b41e4840d81dd2128396a Jul 11 14:49:51.294725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-VolumesNegativeTest-670728480 Jul 11 14:49:51.294725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-VolumesNegativeTest-670728480-project Jul 11 14:49:51.294725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:49:51.294725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:49:51.294725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 47a0b93509ee497e98c3db9947c16e80 Jul 11 14:49:51.294725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-VolumesNegativeTest-670728480-project Jul 11 14:49:51.294725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:49:51.294725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance 4e5e7ae8-7506-45ec-89eb-3e0a454e370d could not be found. Jul 11 14:49:51.294725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': '4e5e7ae8-7506-45ec-89eb-3e0a454e370d'} Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4e5e7ae8-7506-45ec-89eb-3e0a454e370d could not be found. Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:49:51.295798 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:49:51.298113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:49:51.298113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:49:51.298113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:49:51.298113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:49:51.298113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:49:51.298113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:49:51.298113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:49:51.298113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4e5e7ae8-7506-45ec-89eb-3e0a454e370d could not be found. Jul 11 14:49:51.298113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:49:51.298113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:49:51.298113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:49:51.298113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:49:51.298113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:49:51.298113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:49:51.298113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:49:51.298113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:49:51.298113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4e5e7ae8-7506-45ec-89eb-3e0a454e370d could not be found. Jul 11 14:49:51.298113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:49:51.299761 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-dd523309-03d2-4675-8b0f-489b7d61f011 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] Returning 404 to user: Instance 4e5e7ae8-7506-45ec-89eb-3e0a454e370d could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:49:51.299761 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-dd523309-03d2-4675-8b0f-489b7d61f011 tempest-VolumesNegativeTest-670728480 tempest-VolumesNegativeTest-670728480-project] 10.222.0.72 "GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d" status: 404 len: 111 microversion: 2.1 time: 0.056662 Jul 11 14:49:51.299761 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 528/1086] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:49:51 2022] GET /compute/v2.1/servers/4e5e7ae8-7506-45ec-89eb-3e0a454e370d => generated 111 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:50:10.803923 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=20 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:50:10.805301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=44 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:50:10.817412 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-a0b6cfb7-1e4b-4ef0-8951-e54bae778f78 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:10.818574 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a0b6cfb7-1e4b-4ef0-8951-e54bae778f78 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:10.825906 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=36,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:50:10.827902 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=52,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:50:10.834773 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a0b6cfb7-1e4b-4ef0-8951-e54bae778f78 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:10.836034 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a0b6cfb7-1e4b-4ef0-8951-e54bae778f78 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:10.837076 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-69b99e4f-6bd0-4d79-a606-092eb1e232db req-b235e723-9210-49e7-ad51-2d4f54d25d1e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e2f87e01-f360-4b6f-afde-6a64be9f72f7", "tag": "e9f1af2a-fec1-4461-bf84-aca44600c87d"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:50:10.837704 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a0b6cfb7-1e4b-4ef0-8951-e54bae778f78 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:10.839536 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-a0b6cfb7-1e4b-4ef0-8951-e54bae778f78 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:10.848132 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-69b99e4f-6bd0-4d79-a606-092eb1e232db req-b235e723-9210-49e7-ad51-2d4f54d25d1e service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:10.848594 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-69b99e4f-6bd0-4d79-a606-092eb1e232db req-b235e723-9210-49e7-ad51-2d4f54d25d1e service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:10.870091 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-69b99e4f-6bd0-4d79-a606-092eb1e232db req-b235e723-9210-49e7-ad51-2d4f54d25d1e service nova] Creating event network-changed:e9f1af2a-fec1-4461-bf84-aca44600c87d for instance e2f87e01-f360-4b6f-afde-6a64be9f72f7 on hv-cinder-maste Jul 11 14:50:10.879804 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-69b99e4f-6bd0-4d79-a606-092eb1e232db req-b235e723-9210-49e7-ad51-2d4f54d25d1e service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057235 Jul 11 14:50:10.881359 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 529/1087] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:50:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:50:10.889093 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-a0b6cfb7-1e4b-4ef0-8951-e54bae778f78 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Going to try to terminate instance {{(pid=130159) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:50:11.117015 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-5c32a009-127e-480a-965f-f29a756d9cc7 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Action: 'create', calling method: .version_select at 0x7f592b748b80>, body: {"keypair": {"name": "tempest-TestShelveInstance-1256831838", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGuzOiUH+e2ngJcDKI5hWOsn8jtirjLKmypJdHM0qmkAkKuut5+g8rgHIPk3/ONCAjetaQEH6UXGaTqTgJ+FYo5DNBfLqla8UGRoc6tMXvFVp55vnOpZWSYYW4T2zYNReQ=="}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:50:11.161095 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-5c32a009-127e-480a-965f-f29a756d9cc7 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Getting quotas for project 6c2327c73cc445ba97c2ef24ef45e4d5. Resources: {'key_pairs'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:50:11.165700 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-a0b6cfb7-1e4b-4ef0-8951-e54bae778f78 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "DELETE /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 204 len: 0 microversion: 2.1 time: 0.351828 Jul 11 14:50:11.167387 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 559/1088] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:50:10 2022] DELETE /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 11 14:50:11.174014 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-5c32a009-127e-480a-965f-f29a756d9cc7 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Getting quotas for user a7ecfb9570f746a0b1a9c29ff7cd64ef and project 6c2327c73cc445ba97c2ef24ef45e4d5. Resources: {'key_pairs'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:50:11.185549 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-75855dc9-fc7b-411d-be53-a1be2605cfe5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:11.187878 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-75855dc9-fc7b-411d-be53-a1be2605cfe5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:11.217079 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-75855dc9-fc7b-411d-be53-a1be2605cfe5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:11.218747 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-75855dc9-fc7b-411d-be53-a1be2605cfe5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:11.221043 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-5c32a009-127e-480a-965f-f29a756d9cc7 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.335157 Jul 11 14:50:11.222318 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 530/1089] 10.222.0.72 () {60 vars in 1158 bytes} [Mon Jul 11 14:50:10 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:50:11.234954 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-75855dc9-fc7b-411d-be53-a1be2605cfe5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:11.236067 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-75855dc9-fc7b-411d-be53-a1be2605cfe5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:11.657126 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-75855dc9-fc7b-411d-be53-a1be2605cfe5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:11.698710 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-75855dc9-fc7b-411d-be53-a1be2605cfe5 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1577 microversion: 2.1 time: 0.523823 Jul 11 14:50:11.702628 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 560/1090] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:11 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1577 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:50:12.732832 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-ef8109be-0d7d-4842-9fc0-dca682ce427e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:12.742897 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-ef8109be-0d7d-4842-9fc0-dca682ce427e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:12.798543 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ef8109be-0d7d-4842-9fc0-dca682ce427e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:12.810323 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ef8109be-0d7d-4842-9fc0-dca682ce427e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:12.814896 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ef8109be-0d7d-4842-9fc0-dca682ce427e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:12.816093 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ef8109be-0d7d-4842-9fc0-dca682ce427e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:13.180357 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-ef8109be-0d7d-4842-9fc0-dca682ce427e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:13.191167 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-ef8109be-0d7d-4842-9fc0-dca682ce427e tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1577 microversion: 2.1 time: 0.470875 Jul 11 14:50:13.192067 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 531/1091] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:12 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1577 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:50:13.489651 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-9691e4f0-9738-4fa3-9245-c33b99475c15 req-71b8ed5f-9a34-48a3-abcf-47d951880fdf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e2f87e01-f360-4b6f-afde-6a64be9f72f7", "name": "network-vif-deleted", "tag": "e9f1af2a-fec1-4461-bf84-aca44600c87d"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:50:13.517379 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-9691e4f0-9738-4fa3-9245-c33b99475c15 req-71b8ed5f-9a34-48a3-abcf-47d951880fdf service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:13.519903 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-9691e4f0-9738-4fa3-9245-c33b99475c15 req-71b8ed5f-9a34-48a3-abcf-47d951880fdf service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:13.546906 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-9691e4f0-9738-4fa3-9245-c33b99475c15 req-71b8ed5f-9a34-48a3-abcf-47d951880fdf service nova] Creating event network-vif-deleted:e9f1af2a-fec1-4461-bf84-aca44600c87d for instance e2f87e01-f360-4b6f-afde-6a64be9f72f7 on hv-cinder-maste Jul 11 14:50:13.573578 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-9691e4f0-9738-4fa3-9245-c33b99475c15 req-71b8ed5f-9a34-48a3-abcf-47d951880fdf service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.099619 Jul 11 14:50:13.578885 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 561/1092] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:50:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:50:14.222671 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2558db44-c8ce-40c0-ab84-4adee4f47ad2 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:14.224426 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2558db44-c8ce-40c0-ab84-4adee4f47ad2 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:14.235766 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2558db44-c8ce-40c0-ab84-4adee4f47ad2 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:14.237261 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2558db44-c8ce-40c0-ab84-4adee4f47ad2 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:14.240005 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2558db44-c8ce-40c0-ab84-4adee4f47ad2 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:14.241292 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2558db44-c8ce-40c0-ab84-4adee4f47ad2 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:14.299968 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2558db44-c8ce-40c0-ab84-4adee4f47ad2 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:14.300422 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2558db44-c8ce-40c0-ab84-4adee4f47ad2 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:14.300888 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.objects.instance [None req-2558db44-c8ce-40c0-ab84-4adee4f47ad2 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lazy-loading 'fault' on Instance uuid e2f87e01-f360-4b6f-afde-6a64be9f72f7 {{(pid=130160) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:50:14.619947 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2558db44-c8ce-40c0-ab84-4adee4f47ad2 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:14.632673 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2558db44-c8ce-40c0-ab84-4adee4f47ad2 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 200 len: 1185 microversion: 2.1 time: 0.417730 Jul 11 14:50:14.633398 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 532/1093] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:14 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 1185 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:50:15.660118 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-65be749d-dd05-4794-9ab8-cb42278ecc51 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:15.662075 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-65be749d-dd05-4794-9ab8-cb42278ecc51 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: e2f87e01-f360-4b6f-afde-6a64be9f72f7] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:15.676724 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-65be749d-dd05-4794-9ab8-cb42278ecc51 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:15.677588 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-65be749d-dd05-4794-9ab8-cb42278ecc51 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:15.679053 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-65be749d-dd05-4794-9ab8-cb42278ecc51 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:15.679899 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-65be749d-dd05-4794-9ab8-cb42278ecc51 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:15.706042 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-65be749d-dd05-4794-9ab8-cb42278ecc51 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] HTTP exception thrown: Instance e2f87e01-f360-4b6f-afde-6a64be9f72f7 could not be found. Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-65be749d-dd05-4794-9ab8-cb42278ecc51 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Request method failure captured: Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 HTTP/1.1 Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDatkPKLV3gGEijxILG_rdpdg9DW_7qpS_AOU-l0RpuGuc3ntiuQCrsxEAFHVLaJAgdSMGRlHE7aWkLB0HxWtv0M50LGlLeUNDhMrIRsXvtM1cdPVKwP9HSD50o6sO2CtsHMEnTPPPgBI_ekQM6IpcM5Nj1c1ZwFE9vfwHMe24itu1A Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: 3d541b7a72dc4b4494a42bda6fa75026 Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-TestVolumeBootPattern-593501879 Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:50:15.709471 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:50:15.711660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/3d541b7a72dc4b4494a42bda6fa75026"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/3d541b7a72dc4b4494a42bda6fa75026"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_3d541b7a72dc4b4494a42bda6fa75026"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/3d541b7a72dc4b4494a42bda6fa75026"}]}] Jul 11 14:50:15.711660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-TestVolumeBootPattern-593501879 Jul 11 14:50:15.711660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: 3d541b7a72dc4b4494a42bda6fa75026 Jul 11 14:50:15.711660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-TestVolumeBootPattern-593501879 Jul 11 14:50:15.711660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-TestVolumeBootPattern-593501879-project Jul 11 14:50:15.711660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:50:15.711660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:50:15.711660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 0f8e300cd96e446db7430044e69d997a Jul 11 14:50:15.711660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-TestVolumeBootPattern-593501879-project Jul 11 14:50:15.711660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:50:15.711660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance e2f87e01-f360-4b6f-afde-6a64be9f72f7 could not be found. Jul 11 14:50:15.711660 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': 'e2f87e01-f360-4b6f-afde-6a64be9f72f7'} Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e2f87e01-f360-4b6f-afde-6a64be9f72f7 could not be found. Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:50:15.713002 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:50:15.718848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:50:15.718848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:50:15.718848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:50:15.718848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:50:15.718848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:50:15.718848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:50:15.718848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:50:15.718848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e2f87e01-f360-4b6f-afde-6a64be9f72f7 could not be found. Jul 11 14:50:15.718848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:50:15.718848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:50:15.718848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:50:15.718848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:50:15.718848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:50:15.718848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:50:15.718848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:50:15.718848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:50:15.718848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e2f87e01-f360-4b6f-afde-6a64be9f72f7 could not be found. Jul 11 14:50:15.718848 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:50:15.727166 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-65be749d-dd05-4794-9ab8-cb42278ecc51 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Returning 404 to user: Instance e2f87e01-f360-4b6f-afde-6a64be9f72f7 could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:50:15.727166 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-65be749d-dd05-4794-9ab8-cb42278ecc51 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7" status: 404 len: 111 microversion: 2.1 time: 0.061686 Jul 11 14:50:15.727166 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 562/1094] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:15 2022] GET /compute/v2.1/servers/e2f87e01-f360-4b6f-afde-6a64be9f72f7 => generated 111 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:50:19.236868 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-21fdc390-9249-446f-9ce2-8811f44ee8eb req-19fe7065-0085-4ceb-90f9-8c4b4d2dc5ee service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1009499c-a8c8-472c-bef5-fdf42c002c0b", "tag": "1544341d-a8ad-41b0-8bc7-e3d71545d58f"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:50:19.239134 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-706a60da-bef8-49ed-a488-ac6cca469287 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:19.240018 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-706a60da-bef8-49ed-a488-ac6cca469287 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:19.248952 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-21fdc390-9249-446f-9ce2-8811f44ee8eb req-19fe7065-0085-4ceb-90f9-8c4b4d2dc5ee service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:19.249408 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-21fdc390-9249-446f-9ce2-8811f44ee8eb req-19fe7065-0085-4ceb-90f9-8c4b4d2dc5ee service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:19.262600 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-706a60da-bef8-49ed-a488-ac6cca469287 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:19.263142 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-21fdc390-9249-446f-9ce2-8811f44ee8eb req-19fe7065-0085-4ceb-90f9-8c4b4d2dc5ee service nova] Creating event network-changed:1544341d-a8ad-41b0-8bc7-e3d71545d58f for instance 1009499c-a8c8-472c-bef5-fdf42c002c0b on hv-cinder-maste Jul 11 14:50:19.263965 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-706a60da-bef8-49ed-a488-ac6cca469287 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:19.265416 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-706a60da-bef8-49ed-a488-ac6cca469287 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:19.266243 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-706a60da-bef8-49ed-a488-ac6cca469287 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:19.269036 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-21fdc390-9249-446f-9ce2-8811f44ee8eb req-19fe7065-0085-4ceb-90f9-8c4b4d2dc5ee service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035822 Jul 11 14:50:19.270269 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 563/1095] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:50:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:50:19.321737 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-706a60da-bef8-49ed-a488-ac6cca469287 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Going to try to terminate instance {{(pid=130160) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:50:19.541655 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-706a60da-bef8-49ed-a488-ac6cca469287 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "DELETE /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 204 len: 0 microversion: 2.1 time: 0.309000 Jul 11 14:50:19.543914 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 533/1096] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:50:19 2022] DELETE /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 0 bytes in 313 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 11 14:50:19.554666 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-67433b4d-83f1-4033-9bb8-91bcdc40127a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:19.556121 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-67433b4d-83f1-4033-9bb8-91bcdc40127a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:19.569454 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-67433b4d-83f1-4033-9bb8-91bcdc40127a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:19.570318 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-67433b4d-83f1-4033-9bb8-91bcdc40127a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:19.571619 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-67433b4d-83f1-4033-9bb8-91bcdc40127a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:19.572436 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-67433b4d-83f1-4033-9bb8-91bcdc40127a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:19.883434 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-67433b4d-83f1-4033-9bb8-91bcdc40127a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:19.893288 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-67433b4d-83f1-4033-9bb8-91bcdc40127a tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1577 microversion: 2.1 time: 0.343263 Jul 11 14:50:19.893885 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 564/1097] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:19 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1577 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:50:20.920903 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-af0e3326-039c-48d5-a501-5ca24cc9f60b tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:20.922273 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-af0e3326-039c-48d5-a501-5ca24cc9f60b tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:20.938569 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-af0e3326-039c-48d5-a501-5ca24cc9f60b tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:20.940198 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-af0e3326-039c-48d5-a501-5ca24cc9f60b tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:20.942113 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-af0e3326-039c-48d5-a501-5ca24cc9f60b tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:20.945555 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-af0e3326-039c-48d5-a501-5ca24cc9f60b tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:21.155502 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-af0e3326-039c-48d5-a501-5ca24cc9f60b tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:21.165454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-af0e3326-039c-48d5-a501-5ca24cc9f60b tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1577 microversion: 2.1 time: 0.253909 Jul 11 14:50:21.166048 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 534/1098] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:20 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1577 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:50:22.186934 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-676788cd-d054-4d27-880c-73b81b48366c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:22.188506 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-676788cd-d054-4d27-880c-73b81b48366c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:22.200276 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-676788cd-d054-4d27-880c-73b81b48366c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:22.201978 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-676788cd-d054-4d27-880c-73b81b48366c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:22.202984 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-676788cd-d054-4d27-880c-73b81b48366c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:22.203594 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-676788cd-d054-4d27-880c-73b81b48366c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:22.228422 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-13d49a26-2f35-447c-abd3-a8b66460c160 req-b2e7cf68-9eb5-42e0-bff4-d05a2ebfa5d7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1009499c-a8c8-472c-bef5-fdf42c002c0b", "name": "network-vif-deleted", "tag": "1544341d-a8ad-41b0-8bc7-e3d71545d58f"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:50:22.237702 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=4 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:50:22.259338 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-13d49a26-2f35-447c-abd3-a8b66460c160 req-b2e7cf68-9eb5-42e0-bff4-d05a2ebfa5d7 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:22.259953 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-13d49a26-2f35-447c-abd3-a8b66460c160 req-b2e7cf68-9eb5-42e0-bff4-d05a2ebfa5d7 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:22.287013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-13d49a26-2f35-447c-abd3-a8b66460c160 req-b2e7cf68-9eb5-42e0-bff4-d05a2ebfa5d7 service nova] Creating event network-vif-deleted:1544341d-a8ad-41b0-8bc7-e3d71545d58f for instance 1009499c-a8c8-472c-bef5-fdf42c002c0b on hv-cinder-maste Jul 11 14:50:22.296371 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-13d49a26-2f35-447c-abd3-a8b66460c160 req-b2e7cf68-9eb5-42e0-bff4-d05a2ebfa5d7 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071679 Jul 11 14:50:22.297799 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 535/1099] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:50:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:50:22.329951 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-676788cd-d054-4d27-880c-73b81b48366c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '0f8e300cd96e446db7430044e69d997a', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '3d541b7a72dc4b4494a42bda6fa75026', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:22.341032 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-676788cd-d054-4d27-880c-73b81b48366c tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 200 len: 1391 microversion: 2.1 time: 0.158988 Jul 11 14:50:22.341890 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 565/1100] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:22 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 1391 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:50:23.364884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-ec99600b-c94b-4da7-82f2-1cb1d09268aa tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:23.365736 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-ec99600b-c94b-4da7-82f2-1cb1d09268aa tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 1009499c-a8c8-472c-bef5-fdf42c002c0b] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:23.374322 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ec99600b-c94b-4da7-82f2-1cb1d09268aa tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:23.374821 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ec99600b-c94b-4da7-82f2-1cb1d09268aa tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:23.375641 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ec99600b-c94b-4da7-82f2-1cb1d09268aa tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:23.376093 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ec99600b-c94b-4da7-82f2-1cb1d09268aa tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:23.392501 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-ec99600b-c94b-4da7-82f2-1cb1d09268aa tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] HTTP exception thrown: Instance 1009499c-a8c8-472c-bef5-fdf42c002c0b could not be found. Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-ec99600b-c94b-4da7-82f2-1cb1d09268aa tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Request method failure captured: Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b HTTP/1.1 Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDatkPKLV3gGEijxILG_rdpdg9DW_7qpS_AOU-l0RpuGuc3ntiuQCrsxEAFHVLaJAgdSMGRlHE7aWkLB0HxWtv0M50LGlLeUNDhMrIRsXvtM1cdPVKwP9HSD50o6sO2CtsHMEnTPPPgBI_ekQM6IpcM5Nj1c1ZwFE9vfwHMe24itu1A Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: 3d541b7a72dc4b4494a42bda6fa75026 Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-TestVolumeBootPattern-593501879 Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:50:23.394375 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:50:23.396966 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/3d541b7a72dc4b4494a42bda6fa75026"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/3d541b7a72dc4b4494a42bda6fa75026"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_3d541b7a72dc4b4494a42bda6fa75026"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/3d541b7a72dc4b4494a42bda6fa75026"}]}] Jul 11 14:50:23.396966 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-TestVolumeBootPattern-593501879 Jul 11 14:50:23.396966 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: 3d541b7a72dc4b4494a42bda6fa75026 Jul 11 14:50:23.396966 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-TestVolumeBootPattern-593501879 Jul 11 14:50:23.396966 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-TestVolumeBootPattern-593501879-project Jul 11 14:50:23.396966 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:50:23.396966 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:50:23.396966 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 0f8e300cd96e446db7430044e69d997a Jul 11 14:50:23.396966 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-TestVolumeBootPattern-593501879-project Jul 11 14:50:23.396966 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:50:23.396966 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance 1009499c-a8c8-472c-bef5-fdf42c002c0b could not be found. Jul 11 14:50:23.396966 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': '1009499c-a8c8-472c-bef5-fdf42c002c0b'} Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1009499c-a8c8-472c-bef5-fdf42c002c0b could not be found. Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:50:23.398267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:50:23.400647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:50:23.400647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:50:23.400647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:50:23.400647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:50:23.400647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:50:23.400647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:50:23.400647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:50:23.400647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1009499c-a8c8-472c-bef5-fdf42c002c0b could not be found. Jul 11 14:50:23.400647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:50:23.400647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:50:23.400647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:50:23.400647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:50:23.400647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:50:23.400647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:50:23.400647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:50:23.400647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:50:23.400647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1009499c-a8c8-472c-bef5-fdf42c002c0b could not be found. Jul 11 14:50:23.400647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:50:23.403897 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-ec99600b-c94b-4da7-82f2-1cb1d09268aa tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Returning 404 to user: Instance 1009499c-a8c8-472c-bef5-fdf42c002c0b could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:50:23.403897 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-ec99600b-c94b-4da7-82f2-1cb1d09268aa tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b" status: 404 len: 111 microversion: 2.1 time: 0.037238 Jul 11 14:50:23.403897 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 536/1101] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:23 2022] GET /compute/v2.1/servers/1009499c-a8c8-472c-bef5-fdf42c002c0b => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:50:24.008685 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-dd5c5bcb-b996-43dd-9c22-bdcd90c18e2f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:24.009504 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-dd5c5bcb-b996-43dd-9c22-bdcd90c18e2f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:24.017489 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-dd5c5bcb-b996-43dd-9c22-bdcd90c18e2f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:24.017951 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-dd5c5bcb-b996-43dd-9c22-bdcd90c18e2f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:24.018786 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-dd5c5bcb-b996-43dd-9c22-bdcd90c18e2f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:24.019232 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-dd5c5bcb-b996-43dd-9c22-bdcd90c18e2f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:24.035475 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.wsgi [None req-dd5c5bcb-b996-43dd-9c22-bdcd90c18e2f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] HTTP exception thrown: Instance 3a91aed4-52ff-4220-bc8f-571c50452865 could not be found. Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-dd5c5bcb-b996-43dd-9c22-bdcd90c18e2f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Request method failure captured: Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: request: DELETE /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 HTTP/1.1 Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept: application/json Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Accept-Encoding: identity Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Connection: close Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Length: 0 Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Content-Type: application/json Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Host: 10.222.0.72 Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Openstack-System-Scope: None Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: User-Agent: python-urllib3/1.26.9 Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Auth-Token: gAAAAABizDatkPKLV3gGEijxILG_rdpdg9DW_7qpS_AOU-l0RpuGuc3ntiuQCrsxEAFHVLaJAgdSMGRlHE7aWkLB0HxWtv0M50LGlLeUNDhMrIRsXvtM1cdPVKwP9HSD50o6sO2CtsHMEnTPPPgBI_ekQM6IpcM5Nj1c1ZwFE9vfwHMe24itu1A Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Id: None Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Domain-Name: None Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Identity-Status: Confirmed Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Is-Admin-Project: True Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Id: default Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Domain-Name: Default Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Id: 3d541b7a72dc4b4494a42bda6fa75026 Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Project-Name: tempest-TestVolumeBootPattern-593501879 Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Role: member,reader Jul 11 14:50:24.037229 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Roles: member,reader Jul 11 14:50:24.040075 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/3d541b7a72dc4b4494a42bda6fa75026"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/3d541b7a72dc4b4494a42bda6fa75026"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_3d541b7a72dc4b4494a42bda6fa75026"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/3d541b7a72dc4b4494a42bda6fa75026"}]}] Jul 11 14:50:24.040075 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant: tempest-TestVolumeBootPattern-593501879 Jul 11 14:50:24.040075 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Id: 3d541b7a72dc4b4494a42bda6fa75026 Jul 11 14:50:24.040075 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-Tenant-Name: tempest-TestVolumeBootPattern-593501879 Jul 11 14:50:24.040075 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User: tempest-TestVolumeBootPattern-593501879-project Jul 11 14:50:24.040075 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Id: default Jul 11 14:50:24.040075 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Domain-Name: Default Jul 11 14:50:24.040075 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Id: 0f8e300cd96e446db7430044e69d997a Jul 11 14:50:24.040075 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: X-User-Name: tempest-TestVolumeBootPattern-593501879-project Jul 11 14:50:24.040075 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: method: > Jul 11 14:50:24.040075 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: exception: Instance 3a91aed4-52ff-4220-bc8f-571c50452865 could not be found. Jul 11 14:50:24.040075 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: action_args: {'id': '3a91aed4-52ff-4220-bc8f-571c50452865'} Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3a91aed4-52ff-4220-bc8f-571c50452865 could not be found. Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:50:24.041810 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3a91aed4-52ff-4220-bc8f-571c50452865 could not be found. Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3a91aed4-52ff-4220-bc8f-571c50452865 could not be found. Jul 11 14:50:24.045102 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR nova.api.openstack.wsgi Jul 11 14:50:24.047888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-dd5c5bcb-b996-43dd-9c22-bdcd90c18e2f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Returning 404 to user: Instance 3a91aed4-52ff-4220-bc8f-571c50452865 could not be found. {{(pid=130159) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:50:24.047888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-dd5c5bcb-b996-43dd-9c22-bdcd90c18e2f tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "DELETE /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 404 len: 111 microversion: 2.1 time: 0.034055 Jul 11 14:50:24.047888 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 566/1102] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:50:24 2022] DELETE /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Jul 11 14:50:24.048311 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2cc91d8c-213c-4c29-b6d3-3784d576c553 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:24.048663 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2cc91d8c-213c-4c29-b6d3-3784d576c553 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] [instance: 3a91aed4-52ff-4220-bc8f-571c50452865] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:24.055271 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2cc91d8c-213c-4c29-b6d3-3784d576c553 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:24.055725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2cc91d8c-213c-4c29-b6d3-3784d576c553 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:24.056553 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2cc91d8c-213c-4c29-b6d3-3784d576c553 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:24.056994 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2cc91d8c-213c-4c29-b6d3-3784d576c553 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:24.073108 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-2cc91d8c-213c-4c29-b6d3-3784d576c553 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] HTTP exception thrown: Instance 3a91aed4-52ff-4220-bc8f-571c50452865 could not be found. Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2cc91d8c-213c-4c29-b6d3-3784d576c553 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Request method failure captured: Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 HTTP/1.1 Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDatkPKLV3gGEijxILG_rdpdg9DW_7qpS_AOU-l0RpuGuc3ntiuQCrsxEAFHVLaJAgdSMGRlHE7aWkLB0HxWtv0M50LGlLeUNDhMrIRsXvtM1cdPVKwP9HSD50o6sO2CtsHMEnTPPPgBI_ekQM6IpcM5Nj1c1ZwFE9vfwHMe24itu1A Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: 3d541b7a72dc4b4494a42bda6fa75026 Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-TestVolumeBootPattern-593501879 Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:50:24.075419 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:50:24.078613 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/3d541b7a72dc4b4494a42bda6fa75026"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/3d541b7a72dc4b4494a42bda6fa75026"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_3d541b7a72dc4b4494a42bda6fa75026"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/3d541b7a72dc4b4494a42bda6fa75026"}]}] Jul 11 14:50:24.078613 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-TestVolumeBootPattern-593501879 Jul 11 14:50:24.078613 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: 3d541b7a72dc4b4494a42bda6fa75026 Jul 11 14:50:24.078613 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-TestVolumeBootPattern-593501879 Jul 11 14:50:24.078613 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-TestVolumeBootPattern-593501879-project Jul 11 14:50:24.078613 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:50:24.078613 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:50:24.078613 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: 0f8e300cd96e446db7430044e69d997a Jul 11 14:50:24.078613 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-TestVolumeBootPattern-593501879-project Jul 11 14:50:24.078613 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:50:24.078613 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance 3a91aed4-52ff-4220-bc8f-571c50452865 could not be found. Jul 11 14:50:24.078613 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': '3a91aed4-52ff-4220-bc8f-571c50452865'} Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3a91aed4-52ff-4220-bc8f-571c50452865 could not be found. Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:50:24.080033 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:50:24.082699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:50:24.082699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:50:24.082699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:50:24.082699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:50:24.082699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:50:24.082699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:50:24.082699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:50:24.082699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3a91aed4-52ff-4220-bc8f-571c50452865 could not be found. Jul 11 14:50:24.082699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:50:24.082699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:50:24.082699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:50:24.082699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:50:24.082699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:50:24.082699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:50:24.082699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:50:24.082699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:50:24.082699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3a91aed4-52ff-4220-bc8f-571c50452865 could not be found. Jul 11 14:50:24.082699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:50:24.085292 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2cc91d8c-213c-4c29-b6d3-3784d576c553 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Returning 404 to user: Instance 3a91aed4-52ff-4220-bc8f-571c50452865 could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:50:24.085292 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2cc91d8c-213c-4c29-b6d3-3784d576c553 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865" status: 404 len: 111 microversion: 2.1 time: 0.033123 Jul 11 14:50:24.085292 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 537/1103] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:24 2022] GET /compute/v2.1/servers/3a91aed4-52ff-4220-bc8f-571c50452865 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:50:25.540507 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-866d13b7-e288-4c99-abc1-7e7a9a26d756 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] Calling method '.version_select at 0x7f5930137e50>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:25.569191 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-866d13b7-e288-4c99-abc1-7e7a9a26d756 tempest-TestVolumeBootPattern-593501879 tempest-TestVolumeBootPattern-593501879-project] 10.222.0.72 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-2098849656" status: 202 len: 0 microversion: 2.1 time: 0.032979 Jul 11 14:50:25.570047 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 567/1104] 10.222.0.72 () {58 vars in 1262 bytes} [Mon Jul 11 14:50:25 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-2098849656 => generated 0 bytes in 35 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 11 14:50:28.614636 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-824cfdf5-5211-4c63-bbdc-70e8af9a6a5a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-1343459290", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1256831838", "security_groups": [{"name": "tempest-secgroup-smoke-1653795011"}], "networks": [{"uuid": "9ee8f08a-2912-4a1e-94eb-5d8115a4fde3"}], "block_device_mapping_v2": [{"uuid": "66536f1e-ad9a-4e28-8688-9559126077c7", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:50:28.858968 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.network.neutron [None req-824cfdf5-5211-4c63-bbdc-70e8af9a6a5a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] validate_networks() for [('9ee8f08a-2912-4a1e-94eb-5d8115a4fde3', None, None, None, None, None)] {{(pid=130160) validate_networks /opt/stack/nova/nova/network/neutron.py:2626}} Jul 11 14:50:29.609133 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-824cfdf5-5211-4c63-bbdc-70e8af9a6a5a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Flavor limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Jul 11 14:50:29.609822 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-824cfdf5-5211-4c63-bbdc-70e8af9a6a5a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Image limits 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Jul 11 14:50:29.610408 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-824cfdf5-5211-4c63-bbdc-70e8af9a6a5a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Flavor pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Jul 11 14:50:29.610949 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-824cfdf5-5211-4c63-bbdc-70e8af9a6a5a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Image pref 0:0:0 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Jul 11 14:50:29.611474 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.virt.hardware [None req-824cfdf5-5211-4c63-bbdc-70e8af9a6a5a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=130160) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Jul 11 14:50:29.624773 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-824cfdf5-5211-4c63-bbdc-70e8af9a6a5a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:29.626391 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-824cfdf5-5211-4c63-bbdc-70e8af9a6a5a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:29.627551 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-824cfdf5-5211-4c63-bbdc-70e8af9a6a5a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:29.628527 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-824cfdf5-5211-4c63-bbdc-70e8af9a6a5a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:29.666189 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-824cfdf5-5211-4c63-bbdc-70e8af9a6a5a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Getting quotas for project 6c2327c73cc445ba97c2ef24ef45e4d5. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:393}} Jul 11 14:50:29.684776 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.quota [None req-824cfdf5-5211-4c63-bbdc-70e8af9a6a5a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Getting quotas for user a7ecfb9570f746a0b1a9c29ff7cd64ef and project 6c2327c73cc445ba97c2ef24ef45e4d5. Resources: {'ram', 'cores', 'instances'} {{(pid=130160) _get_quotas /opt/stack/nova/nova/quota.py:383}} Jul 11 14:50:29.779459 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-824cfdf5-5211-4c63-bbdc-70e8af9a6a5a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Going to run 1 instances... {{(pid=130160) _provision_instances /opt/stack/nova/nova/compute/api.py:1389}} Jul 11 14:50:29.789074 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-824cfdf5-5211-4c63-bbdc-70e8af9a6a5a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='66536f1e-ad9a-4e28-8688-9559126077c7',volume_size=None,volume_type=None)] {{(pid=130160) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1807}} Jul 11 14:50:30.025367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-824cfdf5-5211-4c63-bbdc-70e8af9a6a5a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.418912 Jul 11 14:50:30.028032 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 538/1105] 10.222.0.72 () {60 vars in 1146 bytes} [Mon Jul 11 14:50:28 2022] POST /compute/v2.1/servers => generated 406 bytes in 1423 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Jul 11 14:50:30.047074 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-46a9f969-f7e2-4ef8-8614-e8fed9821e72 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:30.049723 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-46a9f969-f7e2-4ef8-8614-e8fed9821e72 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:30.221926 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-46a9f969-f7e2-4ef8-8614-e8fed9821e72 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:30.236113 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-46a9f969-f7e2-4ef8-8614-e8fed9821e72 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1089 microversion: 2.1 time: 0.197797 Jul 11 14:50:30.249274 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 568/1106] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:30 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1089 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:50:31.260041 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-cc49431b-bd11-4510-9ba3-1ffca31f7bb0 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:31.261292 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-cc49431b-bd11-4510-9ba3-1ffca31f7bb0 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:31.271544 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc49431b-bd11-4510-9ba3-1ffca31f7bb0 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:31.272149 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc49431b-bd11-4510-9ba3-1ffca31f7bb0 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:31.273109 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc49431b-bd11-4510-9ba3-1ffca31f7bb0 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:31.273805 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cc49431b-bd11-4510-9ba3-1ffca31f7bb0 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:31.374676 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-cc49431b-bd11-4510-9ba3-1ffca31f7bb0 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:31.392367 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-cc49431b-bd11-4510-9ba3-1ffca31f7bb0 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1187 microversion: 2.1 time: 0.137762 Jul 11 14:50:31.394091 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 539/1107] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:31 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1187 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:50:32.428828 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-87e90b7c-4592-4207-aed7-4881a4f39643 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:32.430077 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-87e90b7c-4592-4207-aed7-4881a4f39643 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:32.442514 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-87e90b7c-4592-4207-aed7-4881a4f39643 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:32.443249 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-87e90b7c-4592-4207-aed7-4881a4f39643 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:32.444606 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-87e90b7c-4592-4207-aed7-4881a4f39643 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:32.445339 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-87e90b7c-4592-4207-aed7-4881a4f39643 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:32.628489 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-87e90b7c-4592-4207-aed7-4881a4f39643 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:32.651585 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-87e90b7c-4592-4207-aed7-4881a4f39643 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1193 microversion: 2.1 time: 0.227750 Jul 11 14:50:32.676092 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 569/1108] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:32 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1193 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:50:33.688893 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-28f726a6-eec5-43d6-96b9-059695d28095 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:33.691032 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-28f726a6-eec5-43d6-96b9-059695d28095 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:33.702941 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-28f726a6-eec5-43d6-96b9-059695d28095 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:33.704129 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-28f726a6-eec5-43d6-96b9-059695d28095 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:33.705767 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-28f726a6-eec5-43d6-96b9-059695d28095 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:33.706884 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-28f726a6-eec5-43d6-96b9-059695d28095 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:34.084945 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-28f726a6-eec5-43d6-96b9-059695d28095 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:34.108810 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-28f726a6-eec5-43d6-96b9-059695d28095 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1261 microversion: 2.1 time: 0.427671 Jul 11 14:50:34.123112 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 540/1109] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:33 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1261 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:50:34.210717 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-15791351-4069-4888-b5bf-c7042115afed req-a9f0755c-50a3-437f-9723-afcad721a56c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3a349de6-9b34-4c36-90fb-a350959a34bf", "tag": "a021f0b5-a267-4147-96f3-b6112f84ccb2"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:50:34.225703 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-15791351-4069-4888-b5bf-c7042115afed req-a9f0755c-50a3-437f-9723-afcad721a56c service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:34.226389 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-15791351-4069-4888-b5bf-c7042115afed req-a9f0755c-50a3-437f-9723-afcad721a56c service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:34.255602 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-15791351-4069-4888-b5bf-c7042115afed req-a9f0755c-50a3-437f-9723-afcad721a56c service nova] Creating event network-changed:a021f0b5-a267-4147-96f3-b6112f84ccb2 for instance 3a349de6-9b34-4c36-90fb-a350959a34bf on hv-cinder-maste Jul 11 14:50:34.264791 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-15791351-4069-4888-b5bf-c7042115afed req-a9f0755c-50a3-437f-9723-afcad721a56c service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060229 Jul 11 14:50:34.266545 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 570/1110] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:50:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:50:35.136210 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3c9946bc-bfe1-45e0-9111-d08cd0ef16cf tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:35.137942 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-3c9946bc-bfe1-45e0-9111-d08cd0ef16cf tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:35.150735 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3c9946bc-bfe1-45e0-9111-d08cd0ef16cf tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:35.151949 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3c9946bc-bfe1-45e0-9111-d08cd0ef16cf tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:35.153643 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3c9946bc-bfe1-45e0-9111-d08cd0ef16cf tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:35.154709 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3c9946bc-bfe1-45e0-9111-d08cd0ef16cf tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:35.432600 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-3c9946bc-bfe1-45e0-9111-d08cd0ef16cf tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:35.442172 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-3c9946bc-bfe1-45e0-9111-d08cd0ef16cf tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1261 microversion: 2.1 time: 0.312589 Jul 11 14:50:35.452576 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 541/1111] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:35 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1261 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:50:36.463549 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-84bed972-00b2-4e0b-83dd-ce5a5107d973 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:36.464502 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-84bed972-00b2-4e0b-83dd-ce5a5107d973 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:36.473422 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84bed972-00b2-4e0b-83dd-ce5a5107d973 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:36.474227 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84bed972-00b2-4e0b-83dd-ce5a5107d973 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:36.475664 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84bed972-00b2-4e0b-83dd-ce5a5107d973 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:36.476274 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-84bed972-00b2-4e0b-83dd-ce5a5107d973 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:36.646994 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-84bed972-00b2-4e0b-83dd-ce5a5107d973 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:36.658164 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-84bed972-00b2-4e0b-83dd-ce5a5107d973 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1261 microversion: 2.1 time: 0.201502 Jul 11 14:50:36.670075 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 571/1112] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:36 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1261 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:50:37.681007 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-2a39a0b1-22e9-4d85-9149-434cf2557376 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:37.682061 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-2a39a0b1-22e9-4d85-9149-434cf2557376 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:37.694259 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2a39a0b1-22e9-4d85-9149-434cf2557376 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:37.695094 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2a39a0b1-22e9-4d85-9149-434cf2557376 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:37.696521 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2a39a0b1-22e9-4d85-9149-434cf2557376 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:37.697307 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-2a39a0b1-22e9-4d85-9149-434cf2557376 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:37.853148 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-2a39a0b1-22e9-4d85-9149-434cf2557376 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:37.861881 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-2a39a0b1-22e9-4d85-9149-434cf2557376 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1261 microversion: 2.1 time: 0.187914 Jul 11 14:50:37.862493 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 542/1113] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:37 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1261 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:50:38.879822 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-92541309-f18b-4e8f-9ffa-f069eb302900 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:38.880410 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-92541309-f18b-4e8f-9ffa-f069eb302900 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:38.888145 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-92541309-f18b-4e8f-9ffa-f069eb302900 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:38.888636 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-92541309-f18b-4e8f-9ffa-f069eb302900 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:38.889612 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-92541309-f18b-4e8f-9ffa-f069eb302900 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:38.890087 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-92541309-f18b-4e8f-9ffa-f069eb302900 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:39.039975 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-92541309-f18b-4e8f-9ffa-f069eb302900 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:39.048140 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-92541309-f18b-4e8f-9ffa-f069eb302900 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1261 microversion: 2.1 time: 0.172433 Jul 11 14:50:39.048931 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 572/1114] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:38 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1261 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:50:40.065550 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a68ebd4d-6e63-44ca-8ed8-5193623e8216 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:40.066257 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a68ebd4d-6e63-44ca-8ed8-5193623e8216 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:40.068983 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=2 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:50:40.069949 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell0:SELECT=6 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:50:40.076523 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a68ebd4d-6e63-44ca-8ed8-5193623e8216 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:40.076995 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a68ebd4d-6e63-44ca-8ed8-5193623e8216 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:40.077791 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a68ebd4d-6e63-44ca-8ed8-5193623e8216 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:40.078237 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a68ebd4d-6e63-44ca-8ed8-5193623e8216 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:40.229777 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a68ebd4d-6e63-44ca-8ed8-5193623e8216 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:40.238454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a68ebd4d-6e63-44ca-8ed8-5193623e8216 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1261 microversion: 2.1 time: 0.176821 Jul 11 14:50:40.239050 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 543/1115] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:40 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1261 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:50:41.264983 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-0c31b3f0-4f52-4bb6-9177-1f09f7877e42 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:41.265695 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-0c31b3f0-4f52-4bb6-9177-1f09f7877e42 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:41.268321 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell0:SELECT=2 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:50:41.269036 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=37,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:50:41.275469 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0c31b3f0-4f52-4bb6-9177-1f09f7877e42 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:41.275953 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0c31b3f0-4f52-4bb6-9177-1f09f7877e42 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:41.276750 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0c31b3f0-4f52-4bb6-9177-1f09f7877e42 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:41.277191 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-0c31b3f0-4f52-4bb6-9177-1f09f7877e42 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:41.281619 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=61,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:50:41.444267 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-0c31b3f0-4f52-4bb6-9177-1f09f7877e42 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:41.452984 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-0c31b3f0-4f52-4bb6-9177-1f09f7877e42 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1261 microversion: 2.1 time: 0.192514 Jul 11 14:50:41.453584 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 573/1116] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:41 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1261 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:50:42.469994 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-ad1f3535-9e2f-4530-98ae-0dac771043c3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:42.470843 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-ad1f3535-9e2f-4530-98ae-0dac771043c3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:42.473501 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=60,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:50:42.481379 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ad1f3535-9e2f-4530-98ae-0dac771043c3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:42.481839 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ad1f3535-9e2f-4530-98ae-0dac771043c3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:42.482673 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ad1f3535-9e2f-4530-98ae-0dac771043c3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:42.483117 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-ad1f3535-9e2f-4530-98ae-0dac771043c3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:42.487211 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=75,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:50:42.632890 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-ad1f3535-9e2f-4530-98ae-0dac771043c3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:42.641380 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-ad1f3535-9e2f-4530-98ae-0dac771043c3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1261 microversion: 2.1 time: 0.175289 Jul 11 14:50:42.641962 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 544/1117] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:42 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1261 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:50:43.659864 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-40667d92-073a-41ba-a400-1ccf76e1b40a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:43.660784 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-40667d92-073a-41ba-a400-1ccf76e1b40a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:43.669522 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-40667d92-073a-41ba-a400-1ccf76e1b40a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:43.670046 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-40667d92-073a-41ba-a400-1ccf76e1b40a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:43.671003 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-40667d92-073a-41ba-a400-1ccf76e1b40a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:43.671602 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-40667d92-073a-41ba-a400-1ccf76e1b40a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:43.829374 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-40667d92-073a-41ba-a400-1ccf76e1b40a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:43.838245 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-40667d92-073a-41ba-a400-1ccf76e1b40a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1261 microversion: 2.1 time: 0.182918 Jul 11 14:50:43.838902 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 574/1118] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:43 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1261 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:50:44.064164 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-da9ced8d-0deb-4ded-a271-dbaa0aa0367f req-c23a8f7b-cc79-485b-aade-18131b6896c3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3a349de6-9b34-4c36-90fb-a350959a34bf", "name": "network-vif-plugged", "status": "completed", "tag": "a021f0b5-a267-4147-96f3-b6112f84ccb2"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:50:44.081462 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-da9ced8d-0deb-4ded-a271-dbaa0aa0367f req-c23a8f7b-cc79-485b-aade-18131b6896c3 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:44.082270 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-da9ced8d-0deb-4ded-a271-dbaa0aa0367f req-c23a8f7b-cc79-485b-aade-18131b6896c3 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:44.105362 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-da9ced8d-0deb-4ded-a271-dbaa0aa0367f req-c23a8f7b-cc79-485b-aade-18131b6896c3 service nova] Creating event network-vif-plugged:a021f0b5-a267-4147-96f3-b6112f84ccb2 for instance 3a349de6-9b34-4c36-90fb-a350959a34bf on hv-cinder-maste Jul 11 14:50:44.114681 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-da9ced8d-0deb-4ded-a271-dbaa0aa0367f req-c23a8f7b-cc79-485b-aade-18131b6896c3 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057309 Jul 11 14:50:44.115789 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 545/1119] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:50:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:50:44.865300 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9ab9b3d3-29e9-4f93-bb9c-0be679e2d2d3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:44.865926 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9ab9b3d3-29e9-4f93-bb9c-0be679e2d2d3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:44.873749 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9ab9b3d3-29e9-4f93-bb9c-0be679e2d2d3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:44.874205 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9ab9b3d3-29e9-4f93-bb9c-0be679e2d2d3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:44.875112 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9ab9b3d3-29e9-4f93-bb9c-0be679e2d2d3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:44.875557 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9ab9b3d3-29e9-4f93-bb9c-0be679e2d2d3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:45.030554 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9ab9b3d3-29e9-4f93-bb9c-0be679e2d2d3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:45.038751 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9ab9b3d3-29e9-4f93-bb9c-0be679e2d2d3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1261 microversion: 2.1 time: 0.177989 Jul 11 14:50:45.039357 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 575/1120] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:44 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1261 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:50:46.062952 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-eb995774-686b-4fa7-9ee9-f5f62e32603c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:46.064193 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-eb995774-686b-4fa7-9ee9-f5f62e32603c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:46.075696 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eb995774-686b-4fa7-9ee9-f5f62e32603c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:46.076487 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eb995774-686b-4fa7-9ee9-f5f62e32603c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:46.077895 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eb995774-686b-4fa7-9ee9-f5f62e32603c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:46.078714 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-eb995774-686b-4fa7-9ee9-f5f62e32603c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:46.236987 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-eb995774-686b-4fa7-9ee9-f5f62e32603c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:46.245127 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-eb995774-686b-4fa7-9ee9-f5f62e32603c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1445 microversion: 2.1 time: 0.191672 Jul 11 14:50:46.246635 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 546/1121] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:46 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1445 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:50:46.263815 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-fe4fbfc1-b235-4845-85c1-bfaaf578d31c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:50:46.264750 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-fe4fbfc1-b235-4845-85c1-bfaaf578d31c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:50:46.271595 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fe4fbfc1-b235-4845-85c1-bfaaf578d31c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:46.272049 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fe4fbfc1-b235-4845-85c1-bfaaf578d31c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:46.272859 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fe4fbfc1-b235-4845-85c1-bfaaf578d31c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:46.273292 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fe4fbfc1-b235-4845-85c1-bfaaf578d31c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:46.427544 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-fe4fbfc1-b235-4845-85c1-bfaaf578d31c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:50:46.436456 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-fe4fbfc1-b235-4845-85c1-bfaaf578d31c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1445 microversion: 2.1 time: 0.177738 Jul 11 14:50:46.437230 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 576/1122] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:50:46 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1445 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:50:48.466348 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-51eab24c-7c07-4c4d-ac8e-41b3a3b8612b req-1a8160ae-776b-4054-96f3-0723b6a8b236 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3a349de6-9b34-4c36-90fb-a350959a34bf", "tag": "a021f0b5-a267-4147-96f3-b6112f84ccb2"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:50:48.481041 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-51eab24c-7c07-4c4d-ac8e-41b3a3b8612b req-1a8160ae-776b-4054-96f3-0723b6a8b236 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:50:48.481858 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-51eab24c-7c07-4c4d-ac8e-41b3a3b8612b req-1a8160ae-776b-4054-96f3-0723b6a8b236 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:50:48.496967 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-51eab24c-7c07-4c4d-ac8e-41b3a3b8612b req-1a8160ae-776b-4054-96f3-0723b6a8b236 service nova] Creating event network-changed:a021f0b5-a267-4147-96f3-b6112f84ccb2 for instance 3a349de6-9b34-4c36-90fb-a350959a34bf on hv-cinder-maste Jul 11 14:50:48.503220 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-51eab24c-7c07-4c4d-ac8e-41b3a3b8612b req-1a8160ae-776b-4054-96f3-0723b6a8b236 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043906 Jul 11 14:50:48.504212 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 547/1123] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:50:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:52:19.839107 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 11 14:52:19.839894 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Mon Jul 11 14:52:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 11 14:52:44.306410 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR oslo.messaging._drivers.impl_rabbit [-] [c402650a-e116-4a79-a557-62ed40abb957] AMQP server on 10.222.0.72:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Jul 11 14:52:44.307336 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=7 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:52:44.307765 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=23 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:52:44.312271 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-fc4e1165-4fb9-4024-b374-49d2b1001a34 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:52:44.312809 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-fc4e1165-4fb9-4024-b374-49d2b1001a34 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:52:44.320248 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fc4e1165-4fb9-4024-b374-49d2b1001a34 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:44.320702 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fc4e1165-4fb9-4024-b374-49d2b1001a34 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:44.321494 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fc4e1165-4fb9-4024-b374-49d2b1001a34 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:44.321927 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-fc4e1165-4fb9-4024-b374-49d2b1001a34 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:44.348444 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.objects.instance [None req-fc4e1165-4fb9-4024-b374-49d2b1001a34 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lazy-loading 'flavor' on Instance uuid 3a349de6-9b34-4c36-90fb-a350959a34bf {{(pid=130159) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1099}} Jul 11 14:52:44.435840 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-fc4e1165-4fb9-4024-b374-49d2b1001a34 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "POST /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf/action" status: 202 len: 0 microversion: 2.1 time: 0.131063 Jul 11 14:52:44.436652 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 577/1124] 10.222.0.72 () {60 vars in 1277 bytes} [Mon Jul 11 14:52:44 2022] POST /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf/action => generated 0 bytes in 133 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:52:44.445703 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=7 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:52:44.447205 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=15 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:52:44.453763 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-9cec6fe4-e399-48f2-97c5-a83c7854fb0c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:52:44.455136 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-9cec6fe4-e399-48f2-97c5-a83c7854fb0c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:52:44.466919 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9cec6fe4-e399-48f2-97c5-a83c7854fb0c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:44.467395 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9cec6fe4-e399-48f2-97c5-a83c7854fb0c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:44.468321 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9cec6fe4-e399-48f2-97c5-a83c7854fb0c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:44.469048 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-9cec6fe4-e399-48f2-97c5-a83c7854fb0c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:44.636703 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-9cec6fe4-e399-48f2-97c5-a83c7854fb0c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:52:44.651630 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-9cec6fe4-e399-48f2-97c5-a83c7854fb0c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1570 microversion: 2.1 time: 0.209081 Jul 11 14:52:44.652689 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 548/1125] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:52:44 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1570 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:52:45.680585 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-f72ec472-f9b0-424d-ab47-9a5df59e4eb4 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:52:45.681690 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-f72ec472-f9b0-424d-ab47-9a5df59e4eb4 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:52:45.690906 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f72ec472-f9b0-424d-ab47-9a5df59e4eb4 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:45.691512 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f72ec472-f9b0-424d-ab47-9a5df59e4eb4 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:45.692424 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f72ec472-f9b0-424d-ab47-9a5df59e4eb4 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:45.693170 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-f72ec472-f9b0-424d-ab47-9a5df59e4eb4 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:45.697807 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO oslo.messaging._drivers.impl_rabbit [-] [c402650a-e116-4a79-a557-62ed40abb957] Reconnected to AMQP server on 10.222.0.72:5672 via [amqp] client with port 52094. Jul 11 14:52:45.845550 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-f72ec472-f9b0-424d-ab47-9a5df59e4eb4 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:52:45.854905 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-f72ec472-f9b0-424d-ab47-9a5df59e4eb4 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1570 microversion: 2.1 time: 0.180883 Jul 11 14:52:45.855531 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 578/1126] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:52:45 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1570 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:52:46.883170 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-0114f656-07ef-46ba-b019-f11bae3dcc71 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:52:46.884224 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-0114f656-07ef-46ba-b019-f11bae3dcc71 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:52:46.895934 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0114f656-07ef-46ba-b019-f11bae3dcc71 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:46.896733 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0114f656-07ef-46ba-b019-f11bae3dcc71 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:46.898251 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0114f656-07ef-46ba-b019-f11bae3dcc71 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:46.898998 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0114f656-07ef-46ba-b019-f11bae3dcc71 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:47.052800 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-0114f656-07ef-46ba-b019-f11bae3dcc71 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:52:47.065640 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-0114f656-07ef-46ba-b019-f11bae3dcc71 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1570 microversion: 2.1 time: 0.189590 Jul 11 14:52:47.066710 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 549/1127] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:52:46 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1570 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:52:48.084016 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-6158e0bf-9a0a-4268-a486-d0eb432dd64e tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:52:48.084941 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-6158e0bf-9a0a-4268-a486-d0eb432dd64e tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:52:48.092192 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6158e0bf-9a0a-4268-a486-d0eb432dd64e tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:48.092827 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6158e0bf-9a0a-4268-a486-d0eb432dd64e tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:48.093663 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6158e0bf-9a0a-4268-a486-d0eb432dd64e tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:48.094143 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-6158e0bf-9a0a-4268-a486-d0eb432dd64e tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:48.243327 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-6158e0bf-9a0a-4268-a486-d0eb432dd64e tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:52:48.252025 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-6158e0bf-9a0a-4268-a486-d0eb432dd64e tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1570 microversion: 2.1 time: 0.171856 Jul 11 14:52:48.252607 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 579/1128] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:52:48 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1570 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:52:49.278200 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-3d61bf73-2fa5-42d2-8048-952f1085c28a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:52:49.279058 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-3d61bf73-2fa5-42d2-8048-952f1085c28a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:52:49.286138 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d61bf73-2fa5-42d2-8048-952f1085c28a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:49.286637 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d61bf73-2fa5-42d2-8048-952f1085c28a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:49.287424 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d61bf73-2fa5-42d2-8048-952f1085c28a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:49.287862 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-3d61bf73-2fa5-42d2-8048-952f1085c28a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:49.451660 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-3d61bf73-2fa5-42d2-8048-952f1085c28a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:52:49.460560 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-3d61bf73-2fa5-42d2-8048-952f1085c28a tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1570 microversion: 2.1 time: 0.186369 Jul 11 14:52:49.461155 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 550/1129] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:52:49 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1570 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:52:50.477965 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-169ea102-6969-4108-b8bb-0c6d5eff5adc tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:52:50.478692 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-169ea102-6969-4108-b8bb-0c6d5eff5adc tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:52:50.485657 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-169ea102-6969-4108-b8bb-0c6d5eff5adc tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:50.486190 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-169ea102-6969-4108-b8bb-0c6d5eff5adc tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:50.487072 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-169ea102-6969-4108-b8bb-0c6d5eff5adc tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:50.487540 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-169ea102-6969-4108-b8bb-0c6d5eff5adc tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:50.643709 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-169ea102-6969-4108-b8bb-0c6d5eff5adc tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:52:50.652661 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-169ea102-6969-4108-b8bb-0c6d5eff5adc tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1570 microversion: 2.1 time: 0.178671 Jul 11 14:52:50.653262 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 580/1130] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:52:50 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1570 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:52:51.678786 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-172bfb1f-84b1-400b-ad11-283f4526f7c3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:52:51.679576 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-172bfb1f-84b1-400b-ad11-283f4526f7c3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:52:51.687227 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-172bfb1f-84b1-400b-ad11-283f4526f7c3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:51.687676 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-172bfb1f-84b1-400b-ad11-283f4526f7c3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:51.688453 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-172bfb1f-84b1-400b-ad11-283f4526f7c3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:51.688895 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-172bfb1f-84b1-400b-ad11-283f4526f7c3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:51.838556 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-172bfb1f-84b1-400b-ad11-283f4526f7c3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:52:51.847069 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-172bfb1f-84b1-400b-ad11-283f4526f7c3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1570 microversion: 2.1 time: 0.172873 Jul 11 14:52:51.847659 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 551/1131] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:52:51 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1570 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:52:52.874308 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-22e60d17-3e0b-4d22-890f-b4bce41af15e tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:52:52.875047 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-22e60d17-3e0b-4d22-890f-b4bce41af15e tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:52:52.882289 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-22e60d17-3e0b-4d22-890f-b4bce41af15e tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:52.882912 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-22e60d17-3e0b-4d22-890f-b4bce41af15e tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:52.883846 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-22e60d17-3e0b-4d22-890f-b4bce41af15e tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:52.884383 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-22e60d17-3e0b-4d22-890f-b4bce41af15e tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:53.036110 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-22e60d17-3e0b-4d22-890f-b4bce41af15e tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:52:53.050382 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-22e60d17-3e0b-4d22-890f-b4bce41af15e tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1570 microversion: 2.1 time: 0.180112 Jul 11 14:52:53.051496 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 581/1132] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:52:52 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1570 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:52:54.075019 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-b14e35f4-1475-49cc-b392-c51bf327afd2 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:52:54.075672 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-b14e35f4-1475-49cc-b392-c51bf327afd2 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:52:54.083023 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b14e35f4-1475-49cc-b392-c51bf327afd2 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:54.083516 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b14e35f4-1475-49cc-b392-c51bf327afd2 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:54.084660 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b14e35f4-1475-49cc-b392-c51bf327afd2 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:54.085148 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b14e35f4-1475-49cc-b392-c51bf327afd2 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:54.232335 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-b14e35f4-1475-49cc-b392-c51bf327afd2 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:52:54.240726 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-b14e35f4-1475-49cc-b392-c51bf327afd2 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1570 microversion: 2.1 time: 0.169808 Jul 11 14:52:54.241774 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 552/1133] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:52:54 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1570 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:52:55.258520 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-23aa95e3-ee58-482c-8a2b-fdbb8488c938 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:52:55.259139 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-23aa95e3-ee58-482c-8a2b-fdbb8488c938 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:52:55.266369 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-23aa95e3-ee58-482c-8a2b-fdbb8488c938 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:55.266851 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-23aa95e3-ee58-482c-8a2b-fdbb8488c938 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:55.267627 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-23aa95e3-ee58-482c-8a2b-fdbb8488c938 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:55.268063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-23aa95e3-ee58-482c-8a2b-fdbb8488c938 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:55.431054 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-23aa95e3-ee58-482c-8a2b-fdbb8488c938 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:52:55.439539 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-23aa95e3-ee58-482c-8a2b-fdbb8488c938 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1570 microversion: 2.1 time: 0.185085 Jul 11 14:52:55.440129 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 582/1134] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:52:55 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1570 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:52:56.457508 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a1ccab51-f525-4cc8-aaeb-6cfbe13cef3f tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:52:56.458178 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a1ccab51-f525-4cc8-aaeb-6cfbe13cef3f tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:52:56.466515 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a1ccab51-f525-4cc8-aaeb-6cfbe13cef3f tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:56.467030 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a1ccab51-f525-4cc8-aaeb-6cfbe13cef3f tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:56.467959 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a1ccab51-f525-4cc8-aaeb-6cfbe13cef3f tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:56.468466 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a1ccab51-f525-4cc8-aaeb-6cfbe13cef3f tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:56.624770 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a1ccab51-f525-4cc8-aaeb-6cfbe13cef3f tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:52:56.633746 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a1ccab51-f525-4cc8-aaeb-6cfbe13cef3f tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1570 microversion: 2.1 time: 0.180272 Jul 11 14:52:56.634331 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 553/1135] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:52:56 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1570 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:52:57.651805 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-9a92ef63-195c-40bc-a4bb-d9a5a5c482d8 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:52:57.652525 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-9a92ef63-195c-40bc-a4bb-d9a5a5c482d8 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:52:57.659633 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9a92ef63-195c-40bc-a4bb-d9a5a5c482d8 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:57.660095 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9a92ef63-195c-40bc-a4bb-d9a5a5c482d8 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:57.660882 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9a92ef63-195c-40bc-a4bb-d9a5a5c482d8 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:57.661328 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-9a92ef63-195c-40bc-a4bb-d9a5a5c482d8 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:57.809407 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-9a92ef63-195c-40bc-a4bb-d9a5a5c482d8 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:52:57.817558 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-9a92ef63-195c-40bc-a4bb-d9a5a5c482d8 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1511 microversion: 2.1 time: 0.170190 Jul 11 14:52:57.818140 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 583/1136] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:52:57 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1511 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:52:57.843971 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-0132f8cf-b4a5-4052-955b-544576484643 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:52:57.844800 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-0132f8cf-b4a5-4052-955b-544576484643 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:52:57.851607 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0132f8cf-b4a5-4052-955b-544576484643 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:57.852052 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0132f8cf-b4a5-4052-955b-544576484643 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:57.852831 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0132f8cf-b4a5-4052-955b-544576484643 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:57.853267 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-0132f8cf-b4a5-4052-955b-544576484643 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:57.939253 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR oslo.messaging._drivers.impl_rabbit [None req-0132f8cf-b4a5-4052-955b-544576484643 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [98d090e7-b1eb-41a4-a401-f6b23dec8067] AMQP server on 10.222.0.72:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Jul 11 14:52:58.948466 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO oslo.messaging._drivers.impl_rabbit [None req-0132f8cf-b4a5-4052-955b-544576484643 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [98d090e7-b1eb-41a4-a401-f6b23dec8067] Reconnected to AMQP server on 10.222.0.72:5672 via [amqp] client with port 52126. Jul 11 14:52:58.952531 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-0132f8cf-b4a5-4052-955b-544576484643 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "POST /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf/action" status: 202 len: 0 microversion: 2.1 time: 1.112597 Jul 11 14:52:58.953252 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 554/1137] 10.222.0.72 () {60 vars in 1277 bytes} [Mon Jul 11 14:52:57 2022] POST /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf/action => generated 0 bytes in 1114 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Jul 11 14:52:58.963607 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1a0cfda7-fa96-4b16-8bb1-5e0d9668e126 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:52:58.964287 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1a0cfda7-fa96-4b16-8bb1-5e0d9668e126 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:52:58.971315 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1a0cfda7-fa96-4b16-8bb1-5e0d9668e126 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:58.971775 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1a0cfda7-fa96-4b16-8bb1-5e0d9668e126 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:58.972566 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1a0cfda7-fa96-4b16-8bb1-5e0d9668e126 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:52:58.973011 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1a0cfda7-fa96-4b16-8bb1-5e0d9668e126 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:52:59.153910 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1a0cfda7-fa96-4b16-8bb1-5e0d9668e126 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:52:59.163798 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1a0cfda7-fa96-4b16-8bb1-5e0d9668e126 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1519 microversion: 2.1 time: 0.205549 Jul 11 14:52:59.164640 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 584/1138] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:52:58 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1519 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:53:00.186479 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-b6c8ba07-2499-4371-a473-bcd21ae662e7 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:53:00.187522 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-b6c8ba07-2499-4371-a473-bcd21ae662e7 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:53:00.201315 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b6c8ba07-2499-4371-a473-bcd21ae662e7 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:00.202003 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b6c8ba07-2499-4371-a473-bcd21ae662e7 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:00.203385 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b6c8ba07-2499-4371-a473-bcd21ae662e7 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:00.204092 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-b6c8ba07-2499-4371-a473-bcd21ae662e7 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:00.380297 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-b6c8ba07-2499-4371-a473-bcd21ae662e7 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:53:00.399230 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-b6c8ba07-2499-4371-a473-bcd21ae662e7 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1521 microversion: 2.1 time: 0.220550 Jul 11 14:53:00.400715 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 555/1139] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:53:00 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1521 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:53:00.617938 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [req-82bcd744-a993-479c-a608-451dac4ad7e2 req-a35dbd41-f3d0-4396-8f02-b5c0c355115d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3a349de6-9b34-4c36-90fb-a350959a34bf", "name": "network-vif-unplugged", "status": "completed", "tag": "a021f0b5-a267-4147-96f3-b6112f84ccb2"}]} {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:53:00.630879 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-82bcd744-a993-479c-a608-451dac4ad7e2 req-a35dbd41-f3d0-4396-8f02-b5c0c355115d service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:00.631498 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [req-82bcd744-a993-479c-a608-451dac4ad7e2 req-a35dbd41-f3d0-4396-8f02-b5c0c355115d service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:00.650948 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.compute.server_external_events [req-82bcd744-a993-479c-a608-451dac4ad7e2 req-a35dbd41-f3d0-4396-8f02-b5c0c355115d service nova] Creating event network-vif-unplugged:a021f0b5-a267-4147-96f3-b6112f84ccb2 for instance 3a349de6-9b34-4c36-90fb-a350959a34bf on hv-cinder-maste Jul 11 14:53:00.656930 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [req-82bcd744-a993-479c-a608-451dac4ad7e2 req-a35dbd41-f3d0-4396-8f02-b5c0c355115d service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.123069 Jul 11 14:53:00.657908 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 585/1140] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:53:00 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:53:01.418279 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a9915bb8-29cc-4ca2-b8ae-89e72bbd39ed tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:53:01.419011 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a9915bb8-29cc-4ca2-b8ae-89e72bbd39ed tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:53:01.426045 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a9915bb8-29cc-4ca2-b8ae-89e72bbd39ed tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:01.426517 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a9915bb8-29cc-4ca2-b8ae-89e72bbd39ed tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:01.427291 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a9915bb8-29cc-4ca2-b8ae-89e72bbd39ed tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:01.427725 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a9915bb8-29cc-4ca2-b8ae-89e72bbd39ed tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:01.582496 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a9915bb8-29cc-4ca2-b8ae-89e72bbd39ed tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:53:01.591667 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a9915bb8-29cc-4ca2-b8ae-89e72bbd39ed tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1577 microversion: 2.1 time: 0.177985 Jul 11 14:53:01.592254 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 556/1141] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:53:01 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1577 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:53:02.618589 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-37c44111-82bf-41b9-bdaf-4f4fa7aee7a2 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:53:02.619287 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-37c44111-82bf-41b9-bdaf-4f4fa7aee7a2 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:53:02.626367 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-37c44111-82bf-41b9-bdaf-4f4fa7aee7a2 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:02.626856 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-37c44111-82bf-41b9-bdaf-4f4fa7aee7a2 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:02.627637 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-37c44111-82bf-41b9-bdaf-4f4fa7aee7a2 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:02.628075 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-37c44111-82bf-41b9-bdaf-4f4fa7aee7a2 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:02.777196 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-37c44111-82bf-41b9-bdaf-4f4fa7aee7a2 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:53:02.786933 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-37c44111-82bf-41b9-bdaf-4f4fa7aee7a2 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1577 microversion: 2.1 time: 0.173246 Jul 11 14:53:02.787652 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 586/1142] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:53:02 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1577 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:53:03.805143 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-74277a81-2e80-492d-8bdf-93064c8fdde6 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:53:03.805841 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-74277a81-2e80-492d-8bdf-93064c8fdde6 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:53:03.813152 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-74277a81-2e80-492d-8bdf-93064c8fdde6 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:03.813654 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-74277a81-2e80-492d-8bdf-93064c8fdde6 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:03.814535 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-74277a81-2e80-492d-8bdf-93064c8fdde6 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:03.815014 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-74277a81-2e80-492d-8bdf-93064c8fdde6 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:03.968189 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-74277a81-2e80-492d-8bdf-93064c8fdde6 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:53:03.977144 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-74277a81-2e80-492d-8bdf-93064c8fdde6 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1577 microversion: 2.1 time: 0.176723 Jul 11 14:53:03.977745 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 557/1143] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:53:03 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1577 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:53:04.994335 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-7f3ed965-d484-4735-995e-fe1642a6e9d8 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:53:04.995030 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-7f3ed965-d484-4735-995e-fe1642a6e9d8 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:53:05.002211 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7f3ed965-d484-4735-995e-fe1642a6e9d8 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:05.002793 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7f3ed965-d484-4735-995e-fe1642a6e9d8 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:05.003623 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7f3ed965-d484-4735-995e-fe1642a6e9d8 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:05.004069 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-7f3ed965-d484-4735-995e-fe1642a6e9d8 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:05.159753 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-7f3ed965-d484-4735-995e-fe1642a6e9d8 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:53:05.168489 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-7f3ed965-d484-4735-995e-fe1642a6e9d8 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1577 microversion: 2.1 time: 0.178028 Jul 11 14:53:05.169441 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 587/1144] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:53:04 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1577 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:53:06.186007 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-06ea778b-2d18-4d29-a027-2f9014c88fb9 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:53:06.186843 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-06ea778b-2d18-4d29-a027-2f9014c88fb9 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:53:06.194874 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-06ea778b-2d18-4d29-a027-2f9014c88fb9 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:06.195328 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-06ea778b-2d18-4d29-a027-2f9014c88fb9 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:06.196107 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-06ea778b-2d18-4d29-a027-2f9014c88fb9 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:06.196542 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-06ea778b-2d18-4d29-a027-2f9014c88fb9 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:06.345270 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-06ea778b-2d18-4d29-a027-2f9014c88fb9 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:53:06.354712 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-06ea778b-2d18-4d29-a027-2f9014c88fb9 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1577 microversion: 2.1 time: 0.172585 Jul 11 14:53:06.355307 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 558/1145] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:53:06 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1577 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:53:07.379500 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1557e8c8-ce2b-4a6e-ba9e-1bc0399a400d tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:53:07.380170 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1557e8c8-ce2b-4a6e-ba9e-1bc0399a400d tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:53:07.388932 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1557e8c8-ce2b-4a6e-ba9e-1bc0399a400d tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:07.389603 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1557e8c8-ce2b-4a6e-ba9e-1bc0399a400d tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:07.390645 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1557e8c8-ce2b-4a6e-ba9e-1bc0399a400d tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:07.391075 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1557e8c8-ce2b-4a6e-ba9e-1bc0399a400d tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:07.551218 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1557e8c8-ce2b-4a6e-ba9e-1bc0399a400d tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:53:07.560701 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1557e8c8-ce2b-4a6e-ba9e-1bc0399a400d tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1577 microversion: 2.1 time: 0.185154 Jul 11 14:53:07.561301 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 588/1146] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:53:07 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1577 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:53:08.580216 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-a018c715-ec77-43d1-9494-7ee3484d9d83 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:53:08.581258 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-a018c715-ec77-43d1-9494-7ee3484d9d83 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:53:08.590769 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a018c715-ec77-43d1-9494-7ee3484d9d83 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:08.591436 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a018c715-ec77-43d1-9494-7ee3484d9d83 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:08.592454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a018c715-ec77-43d1-9494-7ee3484d9d83 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:08.593061 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-a018c715-ec77-43d1-9494-7ee3484d9d83 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:08.777870 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-a018c715-ec77-43d1-9494-7ee3484d9d83 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:53:08.786463 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-a018c715-ec77-43d1-9494-7ee3484d9d83 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1577 microversion: 2.1 time: 0.210672 Jul 11 14:53:08.787079 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 559/1147] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:53:08 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1577 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:53:09.812210 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-57dcb46b-264e-40cc-a34e-9827d024d6b3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:53:09.812834 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-57dcb46b-264e-40cc-a34e-9827d024d6b3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:53:09.819566 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-57dcb46b-264e-40cc-a34e-9827d024d6b3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:09.820043 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-57dcb46b-264e-40cc-a34e-9827d024d6b3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:09.820796 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-57dcb46b-264e-40cc-a34e-9827d024d6b3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:09.821230 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-57dcb46b-264e-40cc-a34e-9827d024d6b3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:09.979829 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-57dcb46b-264e-40cc-a34e-9827d024d6b3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:53:09.988444 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-57dcb46b-264e-40cc-a34e-9827d024d6b3 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1577 microversion: 2.1 time: 0.180174 Jul 11 14:53:09.989062 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 589/1148] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:53:09 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1577 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:53:10.567475 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-728024f0-f335-48b5-a26c-5440026c0220 req-9da0dc01-6042-4798-9f69-a18f5cad7986 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3a349de6-9b34-4c36-90fb-a350959a34bf", "name": "network-vif-plugged", "status": "completed", "tag": "a021f0b5-a267-4147-96f3-b6112f84ccb2"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:53:10.583847 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-728024f0-f335-48b5-a26c-5440026c0220 req-9da0dc01-6042-4798-9f69-a18f5cad7986 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:10.585657 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-728024f0-f335-48b5-a26c-5440026c0220 req-9da0dc01-6042-4798-9f69-a18f5cad7986 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:10.609003 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-728024f0-f335-48b5-a26c-5440026c0220 req-9da0dc01-6042-4798-9f69-a18f5cad7986 service nova] Creating event network-vif-plugged:a021f0b5-a267-4147-96f3-b6112f84ccb2 for instance 3a349de6-9b34-4c36-90fb-a350959a34bf on hv-cinder-maste Jul 11 14:53:10.620310 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-728024f0-f335-48b5-a26c-5440026c0220 req-9da0dc01-6042-4798-9f69-a18f5cad7986 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055503 Jul 11 14:53:10.621742 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 560/1149] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:53:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Jul 11 14:53:11.007962 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-1a99307c-861c-48fe-b13e-86649499bdba tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:53:11.009082 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-1a99307c-861c-48fe-b13e-86649499bdba tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:53:11.016897 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1a99307c-861c-48fe-b13e-86649499bdba tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:11.017557 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1a99307c-861c-48fe-b13e-86649499bdba tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:11.018729 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1a99307c-861c-48fe-b13e-86649499bdba tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:11.019302 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-1a99307c-861c-48fe-b13e-86649499bdba tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:11.226656 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-1a99307c-861c-48fe-b13e-86649499bdba tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:53:11.234880 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-1a99307c-861c-48fe-b13e-86649499bdba tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1577 microversion: 2.1 time: 0.231857 Jul 11 14:53:11.235475 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 590/1150] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:53:11 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1577 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:53:12.253606 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-49d0d665-9098-430c-93a5-c6e83a964b88 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:53:12.254421 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-49d0d665-9098-430c-93a5-c6e83a964b88 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:53:12.261775 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-49d0d665-9098-430c-93a5-c6e83a964b88 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:12.262238 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-49d0d665-9098-430c-93a5-c6e83a964b88 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:12.263325 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-49d0d665-9098-430c-93a5-c6e83a964b88 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:53:12.263815 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-49d0d665-9098-430c-93a5-c6e83a964b88 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:53:12.416000 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-49d0d665-9098-430c-93a5-c6e83a964b88 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:53:12.424861 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-49d0d665-9098-430c-93a5-c6e83a964b88 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1564 microversion: 2.1 time: 0.175659 Jul 11 14:53:12.425768 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 561/1151] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:53:12 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1564 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Jul 11 14:54:03.568039 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 11 14:54:03.568826 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Mon Jul 11 14:54:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 11 14:54:26.258134 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-28dbc029-dddd-4a38-9e14-0cd44bd40d29 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:54:26.259065 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-28dbc029-dddd-4a38-9e14-0cd44bd40d29 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:54:26.262699 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_api:SELECT=30 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:54:26.263267 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_api:SELECT=28 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:54:26.263953 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG dbcounter [-] [130159] Writing DB stats nova_cell1:SELECT=89,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=130159) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:54:26.265070 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG dbcounter [-] [130160] Writing DB stats nova_cell1:SELECT=84,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=130160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:54:26.278923 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-28dbc029-dddd-4a38-9e14-0cd44bd40d29 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:54:26.279866 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-28dbc029-dddd-4a38-9e14-0cd44bd40d29 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:54:26.281626 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-28dbc029-dddd-4a38-9e14-0cd44bd40d29 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:54:26.282449 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-28dbc029-dddd-4a38-9e14-0cd44bd40d29 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:54:26.337191 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-28dbc029-dddd-4a38-9e14-0cd44bd40d29 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Going to try to terminate instance {{(pid=130159) delete /opt/stack/nova/nova/compute/api.py:2664}} Jul 11 14:54:26.339619 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-f4b3b134-d11b-4bb2-b4f6-5456f71798cb req-784e6ea1-1c47-48ca-8d29-d227fc35b127 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3a349de6-9b34-4c36-90fb-a350959a34bf", "tag": "a021f0b5-a267-4147-96f3-b6112f84ccb2"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:54:26.350106 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-f4b3b134-d11b-4bb2-b4f6-5456f71798cb req-784e6ea1-1c47-48ca-8d29-d227fc35b127 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:54:26.350647 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-f4b3b134-d11b-4bb2-b4f6-5456f71798cb req-784e6ea1-1c47-48ca-8d29-d227fc35b127 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:54:26.364204 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-f4b3b134-d11b-4bb2-b4f6-5456f71798cb req-784e6ea1-1c47-48ca-8d29-d227fc35b127 service nova] Creating event network-changed:a021f0b5-a267-4147-96f3-b6112f84ccb2 for instance 3a349de6-9b34-4c36-90fb-a350959a34bf on hv-cinder-maste Jul 11 14:54:26.370103 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-f4b3b134-d11b-4bb2-b4f6-5456f71798cb req-784e6ea1-1c47-48ca-8d29-d227fc35b127 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.118303 Jul 11 14:54:26.371035 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 562/1152] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:54:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:54:26.490548 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-28dbc029-dddd-4a38-9e14-0cd44bd40d29 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "DELETE /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 204 len: 0 microversion: 2.1 time: 0.239298 Jul 11 14:54:26.491867 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 591/1153] 10.222.0.72 () {58 vars in 1238 bytes} [Mon Jul 11 14:54:26 2022] DELETE /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 0 bytes in 243 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Jul 11 14:54:26.507317 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-cb1f8b41-0bf2-4267-ab08-2afec10ceb57 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:54:26.508860 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-cb1f8b41-0bf2-4267-ab08-2afec10ceb57 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:54:26.516891 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cb1f8b41-0bf2-4267-ab08-2afec10ceb57 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:54:26.517504 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cb1f8b41-0bf2-4267-ab08-2afec10ceb57 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:54:26.518413 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cb1f8b41-0bf2-4267-ab08-2afec10ceb57 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:54:26.520300 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-cb1f8b41-0bf2-4267-ab08-2afec10ceb57 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:54:26.753965 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.policy [None req-cb1f8b41-0bf2-4267-ab08-2afec10ceb57 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130160) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:54:26.763205 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-cb1f8b41-0bf2-4267-ab08-2afec10ceb57 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1570 microversion: 2.1 time: 0.264688 Jul 11 14:54:26.763913 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 563/1154] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:54:26 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1570 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:54:27.783046 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-c45efc14-9675-45b5-b766-d849978b64e6 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:54:27.783956 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-c45efc14-9675-45b5-b766-d849978b64e6 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:54:27.792063 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c45efc14-9675-45b5-b766-d849978b64e6 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:54:27.792375 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c45efc14-9675-45b5-b766-d849978b64e6 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:54:27.793173 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c45efc14-9675-45b5-b766-d849978b64e6 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:54:27.793612 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-c45efc14-9675-45b5-b766-d849978b64e6 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:54:27.954098 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-c45efc14-9675-45b5-b766-d849978b64e6 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:54:27.962545 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-c45efc14-9675-45b5-b766-d849978b64e6 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1570 microversion: 2.1 time: 0.184700 Jul 11 14:54:27.963131 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 592/1155] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:54:27 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1570 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:54:28.764607 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [req-b8f910a5-3e8f-499c-90c6-e6f62c9e0133 req-3a907c3d-0fac-401b-8ad5-25d0859d2063 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3a349de6-9b34-4c36-90fb-a350959a34bf", "name": "network-vif-deleted", "tag": "a021f0b5-a267-4147-96f3-b6112f84ccb2"}]} {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Jul 11 14:54:28.774658 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-b8f910a5-3e8f-499c-90c6-e6f62c9e0133 req-3a907c3d-0fac-401b-8ad5-25d0859d2063 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:54:28.775108 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [req-b8f910a5-3e8f-499c-90c6-e6f62c9e0133 req-3a907c3d-0fac-401b-8ad5-25d0859d2063 service nova] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:54:28.787463 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.compute.server_external_events [req-b8f910a5-3e8f-499c-90c6-e6f62c9e0133 req-3a907c3d-0fac-401b-8ad5-25d0859d2063 service nova] Creating event network-vif-deleted:a021f0b5-a267-4147-96f3-b6112f84ccb2 for instance 3a349de6-9b34-4c36-90fb-a350959a34bf on hv-cinder-maste Jul 11 14:54:28.793355 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [req-b8f910a5-3e8f-499c-90c6-e6f62c9e0133 req-3a907c3d-0fac-401b-8ad5-25d0859d2063 service nova] 10.222.0.72 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.126377 Jul 11 14:54:28.794119 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 564/1156] 10.222.0.72 () {64 vars in 1317 bytes} [Mon Jul 11 14:54:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Jul 11 14:54:28.985742 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-24d56b04-ccd6-47fa-8c5b-35aebfbcb58f tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:54:28.987989 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.compute.api [None req-24d56b04-ccd6-47fa-8c5b-35aebfbcb58f tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130159) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:54:28.999420 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-24d56b04-ccd6-47fa-8c5b-35aebfbcb58f tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:54:29.000252 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-24d56b04-ccd6-47fa-8c5b-35aebfbcb58f tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:54:29.001695 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-24d56b04-ccd6-47fa-8c5b-35aebfbcb58f tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:54:29.002521 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG oslo_concurrency.lockutils [None req-24d56b04-ccd6-47fa-8c5b-35aebfbcb58f tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130159) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:54:29.100378 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.policy [None req-24d56b04-ccd6-47fa-8c5b-35aebfbcb58f tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a7ecfb9570f746a0b1a9c29ff7cd64ef', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6c2327c73cc445ba97c2ef24ef45e4d5', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=130159) authorize /opt/stack/nova/nova/policy.py:199}} Jul 11 14:54:29.115431 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-24d56b04-ccd6-47fa-8c5b-35aebfbcb58f tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 200 len: 1220 microversion: 2.1 time: 0.138174 Jul 11 14:54:29.116325 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 593/1157] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:54:28 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 1220 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Jul 11 14:54:30.137550 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c37cb8ea-8367-49be-8548-67025abfb395 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '>' {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:54:30.138421 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.compute.api [None req-c37cb8ea-8367-49be-8548-67025abfb395 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [instance: 3a349de6-9b34-4c36-90fb-a350959a34bf] Fetching instance by UUID {{(pid=130160) get /opt/stack/nova/nova/compute/api.py:2920}} Jul 11 14:54:30.145910 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c37cb8ea-8367-49be-8548-67025abfb395 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:54:30.146359 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c37cb8ea-8367-49be-8548-67025abfb395 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:54:30.147184 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c37cb8ea-8367-49be-8548-67025abfb395 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:386}} Jul 11 14:54:30.147626 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG oslo_concurrency.lockutils [None req-c37cb8ea-8367-49be-8548-67025abfb395 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Lock "14f12807-d728-487f-b4ea-df7168086c51" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=130160) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:400}} Jul 11 14:54:30.164291 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.wsgi [None req-c37cb8ea-8367-49be-8548-67025abfb395 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] HTTP exception thrown: Instance 3a349de6-9b34-4c36-90fb-a350959a34bf could not be found. Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c37cb8ea-8367-49be-8548-67025abfb395 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Request method failure captured: Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: request: GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf HTTP/1.1 Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept: application/json Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Accept-Encoding: identity Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Connection: close Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Length: 0 Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Content-Type: application/json Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Host: 10.222.0.72 Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Openstack-System-Scope: None Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: User-Agent: python-urllib3/1.26.9 Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Auth-Token: gAAAAABizDibGUyJN__kdw5bxBW9kksDLCZvnBd7RJKwsLgSQXvBv8M8lZZ28LTKQ_lefYaTcIaZf7MINmYv75aPqSrVm1OFB325a_P-JZju0dIj-NNv7uMSCTj5PknIf-BUPTCMkHzUwkMeLXm0ZwsvkVIr8V8s0TrsgtJs2rDL3YOsdOrFa00 Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Id: None Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Domain-Name: None Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Identity-Status: Confirmed Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Is-Admin-Project: True Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Id: default Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Domain-Name: Default Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Id: 6c2327c73cc445ba97c2ef24ef45e4d5 Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Project-Name: tempest-TestShelveInstance-1069476942 Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Role: member,reader Jul 11 14:54:30.165986 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Roles: member,reader Jul 11 14:54:30.168713 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Service-Catalog: [{"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/identity"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/image"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72:9696/"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/placement"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/6c2327c73cc445ba97c2ef24ef45e4d5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2/6c2327c73cc445ba97c2ef24ef45e4d5"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.72:8080", "publicURL": "http://10.222.0.72:8080/v1/AUTH_6c2327c73cc445ba97c2ef24ef45e4d5"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.72/volume/v3/6c2327c73cc445ba97c2ef24ef45e4d5"}]}] Jul 11 14:54:30.168713 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant: tempest-TestShelveInstance-1069476942 Jul 11 14:54:30.168713 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Id: 6c2327c73cc445ba97c2ef24ef45e4d5 Jul 11 14:54:30.168713 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-Tenant-Name: tempest-TestShelveInstance-1069476942 Jul 11 14:54:30.168713 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User: tempest-TestShelveInstance-1069476942-project Jul 11 14:54:30.168713 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Id: default Jul 11 14:54:30.168713 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Domain-Name: Default Jul 11 14:54:30.168713 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Id: a7ecfb9570f746a0b1a9c29ff7cd64ef Jul 11 14:54:30.168713 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: X-User-Name: tempest-TestShelveInstance-1069476942-project Jul 11 14:54:30.168713 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: method: > Jul 11 14:54:30.168713 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: exception: Instance 3a349de6-9b34-4c36-90fb-a350959a34bf could not be found. Jul 11 14:54:30.168713 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: action_args: {'id': '3a349de6-9b34-4c36-90fb-a350959a34bf'} Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: {{(pid=130160) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2923, in get Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2870, in _get_instance Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2827, in _get_instance_from_cell Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3a349de6-9b34-4c36-90fb-a350959a34bf could not be found. Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Jul 11 14:54:30.170026 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Jul 11 14:54:30.172308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Jul 11 14:54:30.172308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Jul 11 14:54:30.172308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = self._get_server( Jul 11 14:54:30.172308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Jul 11 14:54:30.172308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Jul 11 14:54:30.172308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Jul 11 14:54:30.172308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Jul 11 14:54:30.172308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3a349de6-9b34-4c36-90fb-a350959a34bf could not be found. Jul 11 14:54:30.172308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:54:30.172308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Jul 11 14:54:30.172308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:54:30.172308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Jul 11 14:54:30.172308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Jul 11 14:54:30.172308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Jul 11 14:54:30.172308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Jul 11 14:54:30.172308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Jul 11 14:54:30.172308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3a349de6-9b34-4c36-90fb-a350959a34bf could not be found. Jul 11 14:54:30.172308 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: ERROR nova.api.openstack.wsgi Jul 11 14:54:30.175874 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: DEBUG nova.api.openstack.wsgi [None req-c37cb8ea-8367-49be-8548-67025abfb395 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Returning 404 to user: Instance 3a349de6-9b34-4c36-90fb-a350959a34bf could not be found. {{(pid=130160) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Jul 11 14:54:30.175874 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO nova.api.openstack.requestlog [None req-c37cb8ea-8367-49be-8548-67025abfb395 tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf" status: 404 len: 111 microversion: 2.1 time: 0.035067 Jul 11 14:54:30.175874 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: [pid: 130160|app: 0|req: 565/1158] 10.222.0.72 () {58 vars in 1235 bytes} [Mon Jul 11 14:54:30 2022] GET /compute/v2.1/servers/3a349de6-9b34-4c36-90fb-a350959a34bf => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Jul 11 14:54:30.360067 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: DEBUG nova.api.openstack.wsgi [None req-7a8e8c2f-d044-46cf-af19-8b134a5c7b4c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] Calling method '.version_select at 0x7f5930223d30>' {{(pid=130159) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Jul 11 14:54:30.365377 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: ERROR oslo.messaging._drivers.impl_rabbit [None req-7a8e8c2f-d044-46cf-af19-8b134a5c7b4c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [095d96bc-2c18-43a7-ab13-5db58748ad61] AMQP server on 10.222.0.72:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Jul 11 14:54:31.373924 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO oslo.messaging._drivers.impl_rabbit [None req-7a8e8c2f-d044-46cf-af19-8b134a5c7b4c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] [095d96bc-2c18-43a7-ab13-5db58748ad61] Reconnected to AMQP server on 10.222.0.72:5672 via [amqp] client with port 52226. Jul 11 14:54:31.396509 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO nova.api.openstack.requestlog [None req-7a8e8c2f-d044-46cf-af19-8b134a5c7b4c tempest-TestShelveInstance-1069476942 tempest-TestShelveInstance-1069476942-project] 10.222.0.72 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1256831838" status: 202 len: 0 microversion: 2.1 time: 1.041757 Jul 11 14:54:31.397111 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: [pid: 130159|app: 0|req: 594/1159] 10.222.0.72 () {58 vars in 1253 bytes} [Mon Jul 11 14:54:30 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1256831838 => generated 0 bytes in 1043 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Jul 11 14:55:39.848013 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 11 14:55:39.848686 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Mon Jul 11 14:55:39 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 11 14:57:22.517454 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 11 14:57:22.518384 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Mon Jul 11 14:57:22 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 11 14:57:23.579170 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 11 14:57:23.579831 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Mon Jul 11 14:57:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 11 14:57:24.316114 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 11 14:57:24.316425 dv-cinder-master-3-iscsi devstack@n-api.service[130159]: Mon Jul 11 14:57:24 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 11 14:58:45.609436 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 11 14:58:45.610147 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Mon Jul 11 14:58:45 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!! Jul 11 14:58:59.857788 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Jul 11 14:58:59.858705 dv-cinder-master-3-iscsi devstack@n-api.service[130160]: Mon Jul 11 14:58:59 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) !!!